MU-ID: Multi-user Identification Through Gaits Using ...

10
MU-ID: Multi-user Identification Through Gaits Using Millimeter Wave Radios Xin Yang * , Jian Liu †* , Yingying Chen , Xiaonan Guo , Yucheng Xie * WINLAB, Rutgers University, New Brunswick, NJ, USA The University of Tennessee, Knoxville, Knoxville, TN, USA Indiana University-Purdue University Indianapolis, Indianapolis, IN, USA Email: [email protected], [email protected], [email protected], {xg6, yx11}@iupui.edu § Corresponding Author Abstract—Multi-user identification could facilitate various large-scale identity-based services such as access control, au- tomatic surveillance system, and personalized services, etc. Al- though existing solutions can identify multiple users using cam- eras, such vision-based approaches usually raise serious privacy concerns and require the presence of line-of-sight. Differently, in this paper, we propose MU-ID, a gait-based multi-user identification system leveraging a single commercial off-the-shelf (COTS) millimeter-wave (mmWave) radar. Particularly, MU-ID takes as input frequency-modulated continuous-wave (FMCW) signals from the radar sensor. Through analyzing the mmWave signals in the range-Doppler domain, MU-ID examines the users’ lower limb movements and captures their distinct gait patterns varying in terms of step length, duration, instantaneous lower limb velocity, and inter-lower limb distance, etc. Additionally, an effective spatial-temporal silhouette analysis is proposed to segment each user’s walking steps. Then, the system identifies steps using a Convolutional Neural Network (CNN) classifier and further identifies the users in the area of interest. We implement MU-ID with the TI AWR1642BOOST mmWave sensor and conduct extensive experiments involving 10 people. The results show that MU-ID achieves up to 97% single-person identification accuracy, and over 92% identification accuracy for up to four people, while maintaining a low false positive rate. I. I NTRODUCTION With the thriving growth of the smart Internet of Things (IoT), pervasive user identification brings huge convenience and changes to people’s lifestyle. For instance, smart IoT systems could sense and recognize users to provide various personalized services. User identity-based authority manage- ment systems could keep the user’s privacy and critical data protected. In addition, the prospering smart environment brings eager demands of simultaneous multiple people identification. The capability to recognize multiple users supports large-scale identity-based services, such as security surveillance and smart access control. However, traditional approaches relying on vision-based sensors (e.g., cameras) usually raise privacy fears and concerns. Thus, a solution that can achieve multiple people identification without compromising convenience and privacy is highly desirable. Traditional user identification solutions are usually based on the knowledge of passwords (i.e., texts and graphical pat- terns) or physiological biometrics (e.g., voice, fingerprint, and iris) [4]. However, they all require users’ intervention and are difficult to identify multiple users simultaneously. Moreover, camera-based approaches [12], [14], [19] based on the users’ facial biometrics are proposed to identify single/multiple in- dividuals without requiring extra effort. Nevertheless, these vision-based approaches record sensitive information such as user’s appearance and face, which may violate user’s privacy, especially when deployed in public areas. Recent studies rely on ambient WiFi signals, particularly channel state information (CSI), to identify individuals according to their distinct gait patterns [27], [29], [30] and the unique human physiological and behavioral characteristics inherited from their daily activ- ities [22]. However, the aforementioned WiFi-based solutions all target on identifying a single user in a room as CSI is susceptible to the environmental changes, which restricts the ability to sense and identify multiple targets. In contrast, mmWave radar signals do not share these constraints due to shorter wavelength and stronger directivity, enabling superior multi-user sensing. For instance, Ahmad et al. [3] propose to monitor vital signs of multiple users simultaneously using an FMCW mmWave radar. Knudde et al. [11] use a mmWave radar to track multiple people’s walking trajectory. In this work, we take one step further and propose a passive multi-user identification system, called MU-ID, grounded on the unique human behavior characteristics (i.e., gait pattern) with a single COTS mmWave radar sensor. Comparing with other lower frequency RF signals (e.g., WiFi), the stronger directivity of mmWave radar makes it resistant to the environ- mental noises. Meanwhile, the shorter wavelength of mmWave signals could provide a much higher resolution of its sensing and imaging capability [28], making it an ideal candidate for capturing unique walking gait patterns of multiple users for identification. Notably, we derive the more stable lower-limb gait characteristics so that MU-ID could accommodate users’ diverse upper-limb behaviors (e.g., carrying bags or holding smartphones) in their daily lives without compromising the identification accuracy. The proposed MU-ID has a broad range of application scenarios. For instance, the system could be deployed at the building entrance to identify staffs/residents for the purposes of access control and information statistics. Merchants such as hotels can deploy such a system at the front desk to establish gait profiles for guests and provide personalized services (e.g., turn on favorite TV channels, set preferred room brightness and air conditioner temperature)

Transcript of MU-ID: Multi-user Identification Through Gaits Using ...

Page 1: MU-ID: Multi-user Identification Through Gaits Using ...

MU-ID: Multi-user Identification Through GaitsUsing Millimeter Wave Radios

Xin Yang∗, Jian Liu†∗, Yingying Chen∗§, Xiaonan Guo‡, Yucheng Xie‡∗WINLAB, Rutgers University, New Brunswick, NJ, USA

†The University of Tennessee, Knoxville, Knoxville, TN, USA‡Indiana University-Purdue University Indianapolis, Indianapolis, IN, USA

Email: [email protected], [email protected], [email protected], {xg6, yx11}@iupui.edu§Corresponding Author

Abstract—Multi-user identification could facilitate variouslarge-scale identity-based services such as access control, au-tomatic surveillance system, and personalized services, etc. Al-though existing solutions can identify multiple users using cam-eras, such vision-based approaches usually raise serious privacyconcerns and require the presence of line-of-sight. Differently,in this paper, we propose MU-ID, a gait-based multi-useridentification system leveraging a single commercial off-the-shelf(COTS) millimeter-wave (mmWave) radar. Particularly, MU-IDtakes as input frequency-modulated continuous-wave (FMCW)signals from the radar sensor. Through analyzing the mmWavesignals in the range-Doppler domain, MU-ID examines the users’lower limb movements and captures their distinct gait patternsvarying in terms of step length, duration, instantaneous lowerlimb velocity, and inter-lower limb distance, etc. Additionally,an effective spatial-temporal silhouette analysis is proposed tosegment each user’s walking steps. Then, the system identifiessteps using a Convolutional Neural Network (CNN) classifier andfurther identifies the users in the area of interest. We implementMU-ID with the TI AWR1642BOOST mmWave sensor andconduct extensive experiments involving 10 people. The resultsshow that MU-ID achieves up to 97% single-person identificationaccuracy, and over 92% identification accuracy for up to fourpeople, while maintaining a low false positive rate.

I. INTRODUCTION

With the thriving growth of the smart Internet of Things(IoT), pervasive user identification brings huge convenienceand changes to people’s lifestyle. For instance, smart IoTsystems could sense and recognize users to provide variouspersonalized services. User identity-based authority manage-ment systems could keep the user’s privacy and critical dataprotected. In addition, the prospering smart environment bringseager demands of simultaneous multiple people identification.The capability to recognize multiple users supports large-scaleidentity-based services, such as security surveillance and smartaccess control. However, traditional approaches relying onvision-based sensors (e.g., cameras) usually raise privacy fearsand concerns. Thus, a solution that can achieve multiple peopleidentification without compromising convenience and privacyis highly desirable.

Traditional user identification solutions are usually basedon the knowledge of passwords (i.e., texts and graphical pat-terns) or physiological biometrics (e.g., voice, fingerprint, andiris) [4]. However, they all require users’ intervention and aredifficult to identify multiple users simultaneously. Moreover,

camera-based approaches [12], [14], [19] based on the users’facial biometrics are proposed to identify single/multiple in-dividuals without requiring extra effort. Nevertheless, thesevision-based approaches record sensitive information such asuser’s appearance and face, which may violate user’s privacy,especially when deployed in public areas. Recent studies relyon ambient WiFi signals, particularly channel state information(CSI), to identify individuals according to their distinct gaitpatterns [27], [29], [30] and the unique human physiologicaland behavioral characteristics inherited from their daily activ-ities [22]. However, the aforementioned WiFi-based solutionsall target on identifying a single user in a room as CSIis susceptible to the environmental changes, which restrictsthe ability to sense and identify multiple targets. In contrast,mmWave radar signals do not share these constraints due toshorter wavelength and stronger directivity, enabling superiormulti-user sensing. For instance, Ahmad et al. [3] propose tomonitor vital signs of multiple users simultaneously using anFMCW mmWave radar. Knudde et al. [11] use a mmWaveradar to track multiple people’s walking trajectory.

In this work, we take one step further and propose a passivemulti-user identification system, called MU-ID, grounded onthe unique human behavior characteristics (i.e., gait pattern)with a single COTS mmWave radar sensor. Comparing withother lower frequency RF signals (e.g., WiFi), the strongerdirectivity of mmWave radar makes it resistant to the environ-mental noises. Meanwhile, the shorter wavelength of mmWavesignals could provide a much higher resolution of its sensingand imaging capability [28], making it an ideal candidate forcapturing unique walking gait patterns of multiple users foridentification. Notably, we derive the more stable lower-limbgait characteristics so that MU-ID could accommodate users’diverse upper-limb behaviors (e.g., carrying bags or holdingsmartphones) in their daily lives without compromising theidentification accuracy. The proposed MU-ID has a broadrange of application scenarios. For instance, the system couldbe deployed at the building entrance to identify staffs/residentsfor the purposes of access control and information statistics.Merchants such as hotels can deploy such a system at thefront desk to establish gait profiles for guests and providepersonalized services (e.g., turn on favorite TV channels, setpreferred room brightness and air conditioner temperature)

Page 2: MU-ID: Multi-user Identification Through Gaits Using ...

before the guests arrive at their rooms.In order to build such a robust and deployable multi-user

identification system, we need to address many challenges inpractice. Traditional WiFi-based gait recognition systems [27],[29], [30] identify the user based on the user’s whole-bodymovements, which are also dependent on the user’s irregularupper limbs behaviors (e.g., carrying a bag, and putting handsin the pockets). To mitigate the interferences associated withusers’ upper limbs, in this work, we use a compact low-power frequency-modulated continuous-wave (FMCW) radarwith a narrow vertical field of view (FOV), operating in ahigh frequency band from 77GHz to 81GHz, to only capturethe users’ fine-grained lower limb movements. Moreover,mmWave signals would be influenced by the movements ofboth the user’s left leg and right leg, thus the lower limbof the user may be detected as two different moving targets.To solve this, we design a novel gait feature demonstratingthe user’s lower limb motion in the spatial-temporal domainleveraging the velocity and frequency response. The lowerlimb motion feature reflects multiple users’ step lengths, stepduration, instantaneous limb speeds, and inter-feet distancesto perform multiple people identification. Additionally, oursystem also needs to get rid of the irrelevant signal reflectedfrom static objects (e.g., wall, ceiling), which can be mistak-enly treated as the user in the short time analysis. Hence,we denoise every frame of the radar signals in the range-Doppler domain to eliminate the interference from those staticobjects in the environment. The denoised user-specific gaitprofile is environment independent and can be applied tovarious environments without extra training. In the single-userprofiling phase and multi-user identification phase, we firstapply step segmentation approaches based on peak velocityand silhouette analysis to segment each walking step ofusers. For each step, the unique gait characteristics could beintuitively visualized through a heat map. Therefore, we usea trained deep Convolutional Neural Network (CNN) model,which has been broadly applied for image recognition, toidentify step segments based on visual gait characteristics, andfurther identify the walking users in the area of interest.

The main contributions of our work are summarized asfollows.

• To the best of our knowledge, MU-ID is the first workof performing multi-user identification based on theusers’ lower limb movements using only a single COTSmmWave radar sensor.

• We develop algorithms that have the capability to segmentwalking steps in both single-user and multi-user walkingscenarios satisfying different application requirements.

• Our proposed system extracts unique environment inde-pendent user-specific features that integrate the walkingstep length, duration, instantaneous lower limb velocity,and inter-lower limb distance.

• Our proposed system is capable of mitigating the irrele-vant interference from static objects in the environmentand accurately identifying users in the area of interestgrounded on a fine-tuned deep Convolutional Neural

Network (CNN).• Extensive experiments involving 10 participants in two

environments show that our system can achieve up to 92%accuracy only using 5 walking steps when identifying4 users simultaneously while maintaining a low falsepositive rate.

II. RELATED WORK

User identification systems relying on unique gait patternscan be broadly categorized as:

Sensor-based. Traditional methods mainly utilize varioustypes of sensors, such as cameras, floor sensors (e.g., piezo-electric sensors), and wearable sensors to collect gait infor-mation for human identification. For instance, vision-basedmethods [8], [17], [26] use cameras to capture gait contour im-ages and extract gait information to identify different people.However, these methods may raise privacy concerns and usu-ally require the existence of a line-of-sight (LOS) condition.Ren et al. [21] design a user identification system leveraginggait patterns through run-time accelerometer measurements.Ruben et al. [25] propose a comparative assessment of footstepsignals relying on floor sensors (i.e., piezoelectric sensors)and use the spatiotemporal information of the sensor signalsas a new biometric for human identification. However, theseidentification schemes require users to either carry additionaldevices or install multiple sensors on the floor, which largelyrestricts their application scenarios.

WiFi-based. To overcome the above-mentioned weak-nesses, some researchers seek WiFi-based solutions [22], [27],[29], [30] for gait-based user identification in a small group.For instance, WiWho [29] uses Channel State Information(CSI) of WiFi to extract distinguishing characteristics ofwalking gait for user identification. WifiU [27] identifiesthe user by extracting unique features from spectrograms ofCSI and performing autocorrelation on the torso reflection toremove imperfection in spectrograms. However, these schemesrequire users to walk along a specific path, which is notvery practical in many scenarios. Moreover, Cong et al. [22]utilize WiFi signals to capture unique human physiological andbehavioral characteristics inherited from their daily activities,and further develop a deep-learning-based user authenticationscheme to recognize each individual accurately. However,the aforementioned WiFI-based solutions can only identify asingle user in a room as CSI is susceptible to the environmentalperturbations collected from all directions.

Radar-based. Radar-based human sensing solutions attractconsiderable attention recently. Existing studies propose to usemillimeter-wave radar sensors for gesture recognition [18],[31], indoor localization/tracking [2], [11] and vital signsmonitoring [3], [13]. Additionally, some studies [1], [24] showthat it is possible to leverage radio frequency signals to capturethe human contour and perform human identification. Specif-ically, RF-Capture [1] reconstructs various human body partsand stitches them together through radio frequency signals.It shows the ability to capture a representative human figureand use it to distinguish between various users. Furthermore,

Page 3: MU-ID: Multi-user Identification Through Gaits Using ...

!"#$%#&'()*#+,-&+#

.!)!"#$%#&'(/01+23&'#

451",).&6#7

8#9-'12(/*3&:#;!!<)=1&).&6#7

!"#$%#&'(/)0-,,9#";!!<)=1&).&6#7

*7)>&2#&&3).&6#7

<1?#

!"#$%#&'(

!@ !A

*3B)0323 *3&:#)!!<+

0-,,9#")!!<)*#+%92*3&:#)!!<)*#+%92

0-,,9#")!!<+

>&:9#)!!<)*#+%92

*3&:#)!!<

0-,,9#")

!!<>&:9#)

!!<

C3D

CED

C'D

C6D

C#D

CFD

<G)451", *G)451",

=3&6B1625

.!)H1:&39

H3?#)I#3J+)B125)

01FF#"#&2)I53+#+

*#K#39+)>-> -F)23":#2+)32)25#)

+3?#)"3&:#)3&6)K#9-'12(L

*3&:#;!!<)=1&

.!)!"#$%#&'(/01+23&'#<'

Fig. 1: Illustration of FMCW radar signal processing tech-niques.

Baptist et al. [24] propose to use the Micro-Doppler (MD)signatures retrieved from a low-power radar device to identifypeople based on their gait characteristics. However, there arelimits in these schemas, because they either require the usersto walk along a specific path or fail to recognize more thanone person walking in the same room simultaneously, whichis not very practical in real life.

Different from existing solutions, this paper proposes MU-ID, the first work that has the capability of recognizingmultiple users simultaneously based on lower limb motionfeatures via millimeter-wave radios. We exploit the uniquegait characteristics (e.g., step length, the distance betweentwo lower limbs, and instantaneous lower limb velocity) inthe spatial-temporal domain through mmWave radio analysis.A CNN-based classifier is developed to extract users’ gaitcharacteristics and identify the users’ walking steps, which iscapable of facilitating many user identity-based applicationsand services.

III. BACKGROUND OF FMCW-BASED MMWAVE RADAR

Typically, the walking process involves numerous body partsthat create unique user-specific gait characteristics such as stepdistance, step duration and limb velocity [15], [16]. In thissection, we introduce how the high resolution of a mmWaveradar sensor could capture such unique gait characteristicsby measuring the distance (e.g., step distance, limb position),velocity (e.g., the instantaneous velocity of limbs), and angleof arrival (AoA) necessary to distinguish multiple users.

A. FMCW-based Radar TechniqueThe radio signal travels at the speed of light, and thus,

the time-of-flight (TOF), which is in a picosecond level, isextremely difficult to measure. In this work, we use FMCW-based radar technique that converts the challenge in directlymeasuring TOF to measuring frequency shift instead. In partic-ular, the FMCW-based radar repeatedly transmits continuouschirp signals linearly sweeping through a frequency bandwidthB, and the duration of each chirp signal is Tc, as shown inFigure 1(a). Thus, the slope of the sweep is S = B

Tc. The

received signal is the signal reflected from a target object,which is a delayed version of the transmitted signal. The radar

sensor then uses an integrated mixer generating the subtractionof the transmitted signal and the reflected signal, which isknown as the intermediate frequency (IF) signal. The IF signalis separated by the unit of frame, which contains multiplechirps, to perform frequency domain analysis.

B. Range MeasurementThe range information reflects the user’s location, step

distance, as well as the inter-lower limb distance. To measurethe range of a target, we apply the FFT (i.e., range-FFT) onthe time-domain IF signal. Particularly, when a user appearsin the FOV, the strong frequency response of the human bodyproduces a peak at a specific IF frequency representing theuser’s location. The distance between user and radar can becalculated as follows:

d =fIF · c · Tc

2 ·B=fIF · c2 · S

, (1)

where the c is the speed of light and the frequency of the IFsignal fIF is derived by performing FFT (i.e., range-FFT)on a sliding window with a duration of Tc as shown inFigure 1(b). The result of range-FFT reveals the received fre-quency response at different ranges. Thanks to the centimeter-level range resolution, the range measurement precisely detectsthe position to the limb-level, which enables the measurementof the step length and the inter-lower limb distance.

C. Velocity MeasurementTo further distinguish multiple users via the velocity of their

limb movements, we exploit the phase of the range-FFT toderive the velocity in the Doppler domain. Particularly, thetarget with a moving speed v should have different phases inthe two neighboring range-FFTs, which is associated with amovement distance vTc during a chirp duration. Specifically,the relationship between the phase difference ω betweenneighboring range-FFTs and velocity v is shown as [23]:

v =λ · ω4π · Tc

, (2)

where λ is the wavelength. To distinguish multiple usersusing both range and velocity, another FFT (i.e., Doppler-FFT) is performed along the range-FFT columns as shown inFigure 1(c) to reveal the various speeds of multiple users at thesame range. For instance, the two-colored bins in Figure 1(d)show two targets at the same range with various velocities.

D. Angle of Arrival EstimationThe range measurement only gives the distance of users

in the LOS. Therefore, we use the AoA to depict the exactpositions of users in a spatial Cartesian coordinate system.In particular, we exploit multiple receiving antennas of theFMCW radar to derive the AoA as follows [20]:

θ = sin−1(λ · ω

2π · dinter−Rx), (3)

where dinter−Rx is the distance between neighboring receivingantennas. To further distinguish between the overlap of multi-ple users in the range-Doppler domain, a third FFT (i.e., angle-FFT) is performed along all receiving channels. For instance,after applying angle-FFT on Figure 1(e), we can capture the

Page 4: MU-ID: Multi-user Identification Through Gaits Using ...

!"#$%&'()"*#+

,-%.-(/%00*#$

12"2*(#"-3+4#2%-5%-%#/%+!%)(6"7+"#'+8%#(*0*#$

1*$#"7+,-%.-(/%00*#$+"#'+9%"2:-%+;<2-"/2*(#

=(>%-+=*)?+@(2*(#+

9%"2:-%+;<2-"/2*(#

@:72*&:0%-+4'%#2*5*/"2*(#+!%0:720

))A"6% !"'"-

BCC+@('%7+

D-"*#*#$+

D-"*#*#$+,E"0%

12%.+1%$)%#20+

,-(5*7*#$

4'%#2*5*/"2*(#+,E"0%

@:72*&:0%-+4'%#2*5*/"2*(#+6*"+

F"*2+@('%7

1*7E(:%22%&?"0%'+@:72*&:0%-+

12%.+1%$)%#2"2*(#

@:72*&:0%-+F"*2+9%"2:-%+1%."-"2*(#

9@BA+1*$#"7+

B(77%/2*(#

F"*2+@('%7

8(..7%-&'()"*#+

,-%.-(/%00*#$

Fig. 2: Overview of MU-ID architecture.

AoA of two targets having the same range and velocity, asshown in Figure 1(f).

IV. CHALLENGES AND SYSTEM OVERVIEW

A. ChallengesTo achieve multi-user identification using mmWave radios,

a couple of challenges need to be addressed:Capturing Gait Motions of Multiple Users. Traditional

radar-based gait identification uses Micro-Doppler (MD) fea-tures [6] that can hardly reflect multi-users’ gait patterns.Therefore, salient features should be designed and extractedto capture fine-grained lower limb motions for multi-useridentification.

Separating Steps from Multiple Users. Commonly, mul-tiple users may walk closely together (e.g., side by side, oneafter the other), merging the radar signal reflections of theindividual lower limbs. Therefore, our system should be ableto separate the walking gait of multiple users by step.

Eliminating the Stationary Interference. FMCW radarsignals would be reflected by all the objects in the sensingfield, which makes the signal analysis a challenging taskdue to the severe interferences caused by environmentalnoises and irrelevant reflections (e.g., reflections from walland ceiling). Thus, it is essential to develop an interferenceremoval algorithm that filters out the irrelevant signals so asto extract environment independent gait features for multi-useridentification.

B. System OverviewThe main architecture of MU-ID includes three phases -

signal preprocessing and feature extraction phase, CNN-basedgait model training phase, and multi-user identification phase,as shown in Figure 2. Particularly, the mmWave radar sensortakes the reflected mmWave signals as input and generatesthe IF signal from all receiving antennas (i.e., 4 receivingantennas). To derive the gait characteristics based on the rangeand velocity, we first perform range-FFT and Doppler-FFTon the received signals to reveal the distance and velocity ofmultiple users on a heat map. We then generate an interference

-4 -2 0 2 4Velocity (m/s)

1086420-2-4-6-8

-10

Distanc

e (m

)

−80

−60

−40

−20

Freq

uenc

y Respo

nse

(a) Range-Doppler domain signal.

-4 -2 0 2 4Velocity (m/s)

1086420-2-4-6-8

-10

Distanc

e (m

)

0

10

20

30

Freq

uenc

y Respo

nse

(b) Stationary interference map.

-4 -2 0 2 4Velocity (m/s)

1086420-2-4-6-8

-10

Distanc

e (m

)

−40

−20

0

20

Freq

uenc

y Respo

nse

(c) Interference removal result.

-4 -2 0 2 4Velocity (m/s)

1086420-2-4-6-8

-10

Distanc

e (m

)

0

10

20

30

Freq

uenc

y Respo

nse

(d) Denoised range-Doppler signal.

Fig. 3: Illustration of interference removal and denoising.

distribution map and use a high-pass filter to remove the inter-ference caused by the stationary objects (e.g., wall and ceiling)and further calibrate the received signals. After this step, therange-Doppler map only retains the information associatedwith walking users. To demonstrate the instantaneous speedof users’ lower limbs, we then derive the dominant velocityat every range leveraging the frequency response of the lowerlimb and the corresponding velocity. In the feature extractionstep, we could obtain an intuitive heat map by integrating thevelocity into the spatial-temporal dimension, which representsmultiple users’ distinct lower limb gait motions.

To construct the gait model for each user, we propose asingle-user step segmentation method for the training stageprofiling based on the dominant velocity peaks. The stepsegments reflect the unique gait pattern of each user. Wethen develop a CNN classifier to construct the user-specificenvironment independent gait model. In order to identifymultiple users simultaneously through gaits, we derive a lowerlimb motion feature map that intuitively differentiates multipleusers at distinct ranges. However, the lower limb motionmap has limited abilities to describe multiple users at closeproximity (e.g., walking abreast). To address this, a multi-user gait separation method is proposed to detach the gaitsof multiple users in the range-Doppler domain by exploitingthe AoA information to derive their spatial positions. Tofurther segment individual steps, we use unsupervised learningtechniques that automatically find the optimal strategy basedon step silhouette. The steps are segmented by a fixed-sizewindow and then identified through the gait model trainedwith a developed convolutional neural network.

V. SIGNAL PREPROCESSING AND FEATURE EXTRACTION

A. Range-Doppler-domain PreprocessingTo extract the user-specific gait pattern from the aspect of

range and velocity, MU-ID converts the received raw FMCWsignal into the range-Doppler domain. Particularly, MU-IDfirst performs chirp-wise range-FFT on the received signalto derive the range information of users. A strong frequency

Page 5: MU-ID: Multi-user Identification Through Gaits Using ...

!"#$%&#'(")

!"#$%*+,-"./'

0'"#,12/3#,%&.45%*.6"-'7#

!"#$%&'()%*&+,-$%.

/#(0%&'()%*&+1-23#(4%.

15$-(#(3&6%754-38

Fig. 4: Illustration of the lower limb gait feature.

response Fp is caused by the reflected signal after bouncingoff a reflector at a distance Fpc

2S away from the radar, whereS is the slope of the FMCW signal sweep. It is important tonote that the signals would be reflected by many objects (e.g.,walking users, wall, furniture, and ceiling) in the sensing field,each of which would result in strong frequency response. Tofurther derive the various velocities of these reflectors, we thenapply FFT on frequency-wise range-FFT signals in a frame,which is called Doppler-FFT. Figure 3(a) shows an exampleof the derived Doppler-FFT signal when a user is walkingtowards the radar sensor in a corridor. The x-axis (range-FFTbin) corresponds to the moving speed of the reflectors to theradar, in which zero indicates that the reflector is static. They-axis (Doppler-FFT bin) corresponds to the distance of thereflectors to the radar.

B. Stationary Interference Removal and Denoising

As shown in Figure 3(a), the derived range-Doppler mapcontains the information of moving users, the stationary ob-jects (e.g., wall, furniture, and ceiling), and environmentalmulti-path reflection effects [32]. In order to accurately capturethe users’ walking gait information, we need to eliminate thereflection effects from all of the stationary objects. Our radarsensor is configured to capture the fast-changing lower limbvelocities at a high frame rate (i..e, 100fps). Therefore, forevery frame, the frequency response of the user appears at adifferent range-Doppler position (i.e., distance in Figure 3(a)).In contrast, we find that the frequency response associatedwith the stationary objects in the range-Doppler map remainsconsistent over time. This motivates us to coarsely estimatethe interference from the stationary objects by calculatingthe average frequency response in the range-Doppler domain,which can largely reduce the weights of the user-associatedfrequency response at each range-Doppler position.

Figure 3(b) shows the stationary interference distributionmap derived from a three-second window, which contains300 frames. To remove the interference, MU-ID subtracts theestimated stationary interference (e.g., Figure 3(b)) from eachframe’s range-Doppler domain frequency response (e.g., Fig-ure 3(a)). The range-Doppler map after stationary interferenceremoval is shown in Figure 3(c), which mainly contains thefrequency response caused by walking users.

Additionally, we notice that the massive background noisesaccumulate severe deviations when processing the range-Doppler data. To eliminate the above impacts while preserving

! "!! #!! $!!

%!

"!!

&'()*+,-.*/+012)*3

4(-5*+,-.*/+06278(-9*3

0(3 0:3 093 0.3 0*3

;<(7*+0(3 ;<(7*+0:3 ;<(7*+093 ;<(7*+0.3 ;<(7*+0*3

Fig. 5: Correspondence between the actual step and lower limbfeature map.

(a) Step of User A in Day 1. (b) Step of User A in Day 2.

(c) Step of User B in Day 1. (d) Step of User B in Day 2.

Fig. 6: Comparison of the step segments from two userscollected in different days.

the users’ gait characteristics in the range-Doppler domain, weuse a threshold-based high-pass filter described as below:

R(i,j,k) =

{R(i,j,k), R(i,j,k) ≥ τ,

0, R(i,j,k) < τ,(4)

where R(i,j,k) is the range-Doppler domain frequency re-sponse at the kth frame with range i and velocity j, andthe threshold τ is empirically set to 10dBFS. The denoisedrange-Doppler map is shown as Figure 3(d), which has amuch clearer user-related frequency response comparing toFigure 3(c).

C. Lower Limb Motion Feature ExtractionWe observe that the limited velocity resolution resolves the

velocity of one object at multiple adjacent bins as shownin Figure 3(d). Therefore, we use dominant velocity Vi torepresent the user’s lower limb velocity in every frame, whichcan be calculated as:

Vi =

∑ND

j=1(R(i,j,k)Vj)

ND, i ∈ [1, NR], j ∈ [1, ND]. (5)

where R(i,j,k) is the normalized frequency response; i, j, kstands for the range, velocity, and frame index, respectively;Vj is the velocity corresponding to the the frequency responseR(i,j,k); NR and ND represent the number of range-FFT

Page 6: MU-ID: Multi-user Identification Through Gaits Using ...

Fig. 7: Scatters of twoabreast users in the spa-tial domain.

!"#$%&'($)*

+*$,&-

+*$,&.

Fig. 8: Mixed steps of two abreastusers (left), and separation results(right).

points and Doppler-FFT points, respectively. The dominantvelocity integrates both the frequency response and velocity tocompress a two-dimensional range-Doppler map into a one-dimensional array.

A sequence of the compressed range-velocity data is thenarranged in the order of time, which is regarded as the lowerlimb motion feature map that shows user’s gait characteristics(e.g., step length, duration, inter-lower limb distance and dom-inant lower limb velocity), as shown in Figure 4. To examinewhether our proposed feature reflects the unique gait patternof lower limb, we illustrate the correspondence between theactual gait process, which is captured by a camera, and thelower limb motion feature in Figure 5. The distance changeover time reflects the step length and inter-lower limb distance;the step duration is indicated by the number of frames neededto finish the movement; the instantaneous velocity of the lowerlimb is represented by the dominant velocity and shown as thecolor map. Figure 6 shows the step segments from two userscaptured in two different days. The comparison shows that userA has a faster instantaneous lower limb velocity and a shorterstep length comparing with user B; meanwhile the stepsbelonging to the same user shows high consistency. Thus, thespatial-temporal lower limb motion feature accurately reflectsthe user-specific lower limb gait pattern.

VI. MULTI-USER IDENTIFICATIONThe crucial point in achieving multi-user identification is

to detach the steps from simultaneously walking users. Inthis section, we introduce a lower limb motion separationtechnique and a silhouette-based multi-user step segmentationmethod leveraging unsupervised learning.A. Lower Limb Motion Separation for Multiple Users

The spatial-temporal feature inherently differentiates mul-tiple users at distinct ranges. However, such feature has anintrinsic limitation in distinguishing abreast walking users.Towards this end, we propose a lower limb gait featureseparation approach exploiting the AoA. To detect the exis-tence of abreast users, we empirically infer the range-Dopplerdomain frequency response summation of a single user asRsingle. If the summation of frequency response at a specificrange significantly exceeds the Rsingle, MU-ID confirms theexistence of abreast users and perform the separation.

In order to separate the abreast users, we leverage users’distinct spatial positions. Specifically, we project the data

(a) Lower limb motion map of twousers walking towards the radar.

(b) Step segmentation clustering resultwith step center marked as red.

Fig. 9: Illustration of multi-user step segmentation.

point R(i,j,k) in the kth range-Doppler frame with rangeindex i and velocity index j to a point R(i,j,k) in the two-dimensional spatial Cartesian coordinate system, which takesthe position of the radar as the origin. Leveraging R(i,j,k)’srange Di and angle of arrival ω(i,j,k), the spatial coordinates(XR(i,j,k)

, YR(i,j,k)) of R(i,j,k) is derived as below:{XR(i,j,k)

=Di · sin(ω(i,j,k)),

YR(i,j,k)=Di · cos(ω(i,j,k)).

(6)

To differentiate the attribution of the scatters in the Carte-sian coordinate system, we use the K-means clustering al-gorithm [7] to separate the scatters as shown in Figure 7.To determine the K (i.e., the number of users), MU-IDevaluates the value of K in [1,Kmax] using Calinski-HarabaszIndex [5], where the Kmax is estimated based on the averagenumber of frequency response peaks. We then generate eachuser’s individual gait feature in the range-Doppler domain bymasking the frequency response from other users based onthe spatial clustering results. Figure 8 shows the separationresults of two abreast walking users that correctly identifiedby MU-ID.B. Multi-user Step Segmentation based on Silhouette Analysis

In order to retain the user-specific step length and stepduration features, we perform segmentation using a fixed-sizewindow, which directly crops the step from the gait featuremap. For most of the cases, the users’ gait motions can berecognized from the spatial-temporal feature map, as shownin Figure 9(a). Therefore, the step segmentation needs to becapable of using for both single-user and multi-user scenarios.To achieve this, we use unsupervised learning technique toautomatically detect the silhouette of steps and locate the stepcenter for segmentation.

In order to make the silhouette of steps clearer, we first per-form binarization to convert the lower limb motion map intoa scatter plot. Particularly, the binarization uses a threshold ηto set the data point F(x,y) in the feature map at frame x anddistance y with a lower value as zero, and keep the one with ahigher value as one. The threshold η can be selected throughempirical studies. The binarized scatter plot is derived as:

F(x,y) =

{1, F(x,y) ≥ η,0, F(x,y) < η.

(7)

Afterward, MU-ID categorizes the scatters using density-based spatial clustering of applications with noise (DBSCAN)algorithm. Based on the optimal clustering, we calculate the

Page 7: MU-ID: Multi-user Identification Through Gaits Using ...

!"#$%

!""#!""

&'"(

$%#$%#&%

&'"(

''#''#('

&'"(

%"#%"#!%)

&'"(

)#)#%*(

)&

%*(

*$%#$%

!"

+,-.#''/ +,-.#''/ +,-.#''/

+,-.#''/

0/,%%1"

Fig. 10: Illustration of the proposed CNN structure.

! "#! #!! $#! %!!!

&'()*+,-.*/+012)*3

4256(-7*+0)3

8779)9:(6*.+

4;)2-(-6+<*:;726=

!

"#

#!

$#

"

>

?

@

%!

Fig. 11: Illustration of step segments profiling based ondominant velocity peaks.

coordinates of the cluster center to segment the steps usinga fixed-size window. For a step cluster C contains N pointsD1, D2, · · · , DN , the coordinates of step center (XPc

, YPc) is

derived as: XPc =

∑Ni=1XDi

N,

YPc=

∑Ni=1 YDi

N.

(8)

Figure 9(b) shows the step segmentation results of two userswalking simultaneously with the step center marked as red.

VII. GAIT PROFILE CONSTRUCTION AND CNN-BASEDMODEL TRAINING

A. Step Segments Profiling based on Dominant Velocity PeaksTo efficiently construct the user’s gait profile at the training

phase, we propose an intuitive and efficient single-user stepsegmentation method. Specifically, we locate each step fromthe lower limb gait map based on the observation that thearea of the step center (i.e., the intersection where two lowerlimbs meet) possesses the highest dominant velocity. To derivethe position of the step centers, MU-ID first calculates theaccumulated dominant velocity as Figure 11 shows. The frameindex of the step center corresponds to a peak in the accu-mulated dominant velocity. Through an efficient peak findingalgorithm, MU-ID obtains the position of every step in thetemporal axis first. Then a sliding window-based searchingalgorithm is used to locate the step in the spatial axis. To re-move the fake peaks generated by the peak finding algorithm,MU-ID sets a minimum accumulated dominant velocity as athreshold to avoid invalid gait profiles.

B. CNN Model Training Through Gait ProfilesCNN is widely applied in the image recognition field for its

ability in extracting fine-grained image patterns. To obtain the

!"#$%&'%%() *"#$%&'%%()

$+!,-./0112*

!(3(4

56$,77789:

5(&(#6(;&<4'#6(43

(a) AWR1642BOOST radarpaired with DCA1000EVM datacapture card.

!"#$ !"#

$%&%'

(b) Illustration of the radar displace-ment and possible walk paths.

Fig. 12: Illustration of the experimental setup.

unique gait patterns from the two-dimensional step segments,we develop and fine-tune a deep CNN-based classifier. Weillustrate the structure of MU-ID’s CNN classifier in Fig-ure 10. Particularly, the CNN takes as input the normalized100 × 100px step segments. Next, four convolutional layersvarying in the number and size of convolution kernels extractthe image-based gait patterns. Rectified Linear Unit (ReLU)is used as the activation function to reduce the vanishinggradient, followed by the max-pooling layer to downsize theintermediate output. The output of the last convolutional layeris flattened and processed by a fully connected layer with256 neurons. Finally, a softmax layer outputs the probabilitydistribution as the multi-user identification result.

VIII. PERFORMANCE EVALUATION

A. Experimental Methodology1) Experimental Setup: We implement MU-ID on a

COTS AWR1642BOOST [9] mmWave radar paired with aDCA1000EVM [10] data capture card as shown in Fig-ure 12(a). The radar operates in a frequency band from77GHz to 81GHz with a 4mm wavelength. The radar hastwo transmitting antennas and four receiving antennas linedup horizontally to give a sensing FOV of 120° in the E-plane and 30° in the H-plane, with 15° angle resolution. Theradar is configured to sample 100 frames per second with17 chirps per frame. The chirp slope is 18.83MHz/µs, andeach chirp contains 598 data samples. The ADC sampling rateis 3000k samples per second. The above configuration givesan 18m maximum detectable range with 4cm resolution and8km/h maximum detectable velocity with 2km/h resolution.We displace the radar vertically on the floor pointing to thelower limbs. The evaluation is mainly performed in a 4m×8mlobby. We also test our system in another environment (i.e., a1.6m× 7m corridor) to evaluate the impact of environmentalchanges. The CNN-based classifier is based on TensorFlowframework, the model training and prediction is accomplishedon a desktop server with GPU acceleration capability.

2) Data Collection: We recruit 10 participants aging from21 to 34 with various heights from 164cm to 182cm andweights from 45kg to 74kg. During the data collection process,we use a camera to record the ground truth and label thedataset. The radar is placed by the wall at the center of thelobby edge as demonstrated in Figure 12(b). Users walk fromthe opposite edge of the lobby and are allowed to walk fromvarious starting points (within the FOV) towards the radar. The

Page 8: MU-ID: Multi-user Identification Through Gaits Using ...

0.98

0.00

0.00

0.01

0.00

0.01

0.00

0.01

0.00

0.00

0.01

0.98

0.01

0.00

0.01

0.00

0.00

0.00

0.00

0.00

0.00

0.00

0.99

0.02

0.00

0.00

0.00

0.00

0.00

0.00

0.00

0.00

0.00

0.97

0.00

0.00

0.00

0.00

0.00

0.00

0.00

0.00

0.00

0.00

0.96

0.00

0.00

0.00

0.00

0.00

0.01

0.00

0.00

0.01

0.01

0.94

0.00

0.00

0.00

0.00

0.00

0.00

0.00

0.00

0.00

0.03

1.00

0.00

0.00

0.00

0.00

0.01

0.00

0.00

0.00

0.01

0.00

0.99

0.00

0.00

0.00

0.01

0.00

0.00

0.01

0.00

0.00

0.00

0.98

0.03

0.00

0.00

0.00

0.00

0.01

0.00

0.00

0.00

0.02

0.97

1 2 3 4 5 6 7 8 9 10

Predicted User

1

2

3

4

5

6

7

8

9

10

Actu

al

User

Fig. 13: Confusion matrix ofsingle-user identification.

1 2 3 4 5

Group Id

0

0.2

0.4

0.6

0.8

1

Iden

tifi

cati

on

Acc

ura

cy

1 Step

5 Steps

Fig. 14: Average accuracy ofidentifying two abreast users.

radar records 10 seconds, therefore the path lengths fluctuatein the range from 5m to 7m based on the users’ walk speeds.

Each participant is required to walk a total of eight timesfor the purposes of gait model construction and single-useridentification. The data is collected under varying circum-stances, including time of day, walk speed, and participantoutfit. Unless mentioned otherwise, we split the dataset bya 6/4 ratio, i.e., 60% of the data are used to construct gaitmodels, and 40% are used to evaluate the performance.

For the multi-user scenario, ten participants are randomlysplit into groups with up to four members to walk simul-taneously. Every group member is first required to keep areasonable distance (e.g., 0.3m) from other users to performthe experiments. Additionally, we evaluate the abreast walkingscenario to demonstrate the performance under extreme cases.For every group size, we collect 20 sets of data. Apart fromidentifying multiple people based on a single step, we alsoevaluate the identification performance based on continuousthree-step and five-step majority vote. Explicitly, when themajority vote ties, we specify the identification result to be theone with the highest average confidence. In total, we collectedover 3,000 steps from the ten participants for evaluation.

3) Evaluation Metrics: To quantify the performance of oursystem, we define the following evaluation metrics:

Confusion Matrix. Each row in the confusion matrix is theuser identity ground truth; each column represents the identitypredicted by MU-ID. The entry of ith row and jth columnshows the percentage of user i being identified as user j.

Identification Accuracy: The identification accuracy isdefined as the probability of a user being correctly recognizedby MU-ID.

False Positive Rate (FP rate): FP rate is defined as the ratioof other users (excluding user A) being mistakenly identifiedas user A.

B. Evaluation of Single-user IdentificationWe first examine the identification performance of MU-ID

under the single-user scenario. For the single-user case, weidentify the participants based on one step. Figure 13 showsthe confusion matrix of single-user identification. Particularly,the average identification accuracy achieves 97.7% with astandard deviation of 1.6%. Among all the participants, thelowest identification accuracy achieves 94%. Additionally, thecorresponding average FP rate is 0.2%, and the highest FP

2 3 4

Number of Users per Group

0

0.2

0.4

0.6

0.8

1

Iden

tifi

cati

on

Acc

ura

cy

1 Step

3 Steps

5 Steps

(a) Average accuracy of 1, 3, and 5-step identifiction.

2 3 4

Number of Users per Group

0

0.05

0.1

0.15

0.2

FP

Rate

1 Step

3 Steps

5 Steps

(b) Average FP rate of 1, 3, and 5-stepidentifiction.

Fig. 15: Performance of multi-user identification.

rate is 0.5%. This result validates that our system can identifyindividually walking user with high accuracy.

C. Evaluation of Multi-user IdentificationWe then evaluate the performance of multi-user identifica-

tion. Figure 15(a) depicts the identification accuracy for upto four users walking simultaneously. Apart from the single-step performance, we demonstrate the performance whenidentifying three and five continuous steps.

For two-user identification, the average accuracy achieves91% by identify a single step. When we apply majority voteon continuous three and five steps, the accuracy rises to 97.7%and 100% respectively. The FP rates for two-user identificationare all below 1% as Figure 15(b) shows.

We then evaluate the three-user and four-user scenarios,where the average single-step accuracy descends to 84% and74% respectively, and the average FP rates are around 4%and 9% respectively. The three-step majority vote slightlyincreases the accuracy to 89% and 77% for the three-user andfour-user scenario respectively. When we identify continuousfive steps, the accuracy dramatically rises to 96% for three-user identification and 92% for four-user identification. Theabove results indicate that MU-ID is capable of identifyingmultiple users simultaneously. Moreover, we find that thethree-step majority vote is more sensitive to the misclassifiedsteps, whereas the five-step majority vote accurately identifiesmultiple users.

We then evaluate the case when two users walk towardsthe radar side by side. We randomly combine two participantsto walk abreast; a total of five combinations of participantsare formed. Figure 14 shows the results of abreast useridentification. When identifying a single step, MU-ID showsan average accuracy of 70%. With the help of the five-stepmajority vote, the accuracy increases to 84%. Consideringthat we only use a single mmWave radar and the angularresolution is limited 15°, such accuracy of simultaneous multi-user identification is very encouraging.

D. Impact of Various Factors1) Impact of Train/Test Split Ratio: We change the number

of training data used for gait model construction to examinethe performance of MU-ID under various train/test split ratios.To control other factors that could affect the identificationaccuracy, we evaluate the performance based on the single-user dataset. Each train/test split ratio is evaluated for five

Page 9: MU-ID: Multi-user Identification Through Gaits Using ...

1/9 2/8 3/7 4/6 5/5 6/4 7/3 8/2 9/1

Train/Test Split Ratio

0.5

0.6

0.7

0.8

0.9

1A

ver

age

Iden

tifi

cati

on

Acc

ura

cy

Fig. 16: Error bar of varioustrain/test split ratios.

1 2 3 4 5

User Id

0

0.2

0.4

0.6

0.8

1

Iden

tifi

cati

on

Acc

ura

cy

Natural Walk

Handbag

Backpack

Hands in Pockets

Fig. 17: Identification accu-racy with different upper limbbehaviors.

times to calculate the average accuracy and deviation. Wedemonstrate the results in Figure 16. The results show thatMU-ID has robust identification performance under varioustrain/test split ratios. In particular, our model achieves over96% average accuracy when the training set is no smaller thanthe test set. Even with the train/test ratio of 2/8, our systemstill retains an average accuracy above 90%. Meanwhile, thehighest deviation is around 2.2% for the 1/9 train/test splitratio, which shows the performance of MU-ID is stable. Noticethat even straight-line trajectories have variations across allrounds of tests, therefore our model is unlikely to be overfit.This result indicates that the features extracted by MU-IDshow generality in represents the users’ gait characteristics.Besides, MU-ID requires minimal training process to obtainthe gait features.

2) Impact of Different Environments: We evaluate the iden-tification performance under a non-training environment tovalidate that the gait feature is user-specific and insensitiveto environmental changes. Specifically, we involve five partic-ipants to first train MU-ID with the dataset under the lobbyenvironment. We then evaluate the identification performanceunder a new environment (i.e., a corridor having a size of1.6m×7m) based on the gait models derived under the lobbyenvironment. Afterward, we exchange the training/testing en-vironment. As Figure 18 shows, all of the involved participantscan be recognized in the new environment. Specifically, whenMU-ID identifies the participants under the corridor environ-ment with gait models trained with the lobby dataset, theaverage accuracy and FP rate is 78% and 5% respectively.Exchanging the training/testing environment gives an averageof 74% accuracy and 4% FP rate. Although User 2 and User5 show below-average exception, most users achieve above70% accuracy under a new environment. This result indicatesthat even without retraining the system, MU-ID still identifiesmost of the users’ steps under a new environment. Notice thatsuch results are based on identifying a single step, by applyingthe majority vote on continuous steps, higher identificationaccuracy is promising.

3) Impact of Upper Limb Behaviors: Finally, we study theidentification performance when participants perform variousupper limb behaviors to examine MU-ID’s robustness underpractical daily life scenarios. Five participants are requiredto perform three upper-limb behaviors while walking, i.e., 1)carrying a backpack, 2) grabbing a handbag with one hand,

1 2 3 4 5

User Id

0

0.2

0.4

0.6

0.8

1

Iden

tifi

cati

on

Acc

ura

cy

Test Lobby on Lobby Model

Test Corridor on Lobby Model

Test Corridor on Corridor Model

Test Lobby on Corridor Model

(a) Identification accuracy under dif-ferent environments.

1 2 3 4 5

User Id

0

0.05

0.1

0.15

0.2

FP

Rate

Test Lobby on Lobby Model

Test Corridor on Lobby Model

Test Corridor on Corridor Model

Test Lobby on Corridor Model

(b) FP Rate under different environ-ments.

Fig. 18: Impact of different environments.

3) putting both hands in their trouser pockets. We presentthe identification results in Figure 17. The average accuracyfor grabbing a handbag, carrying a backpack, and with handsin the pockets is 88%, 95%, and 96% respectively. Exceptfor the handbag scenario, which shows a slight decrease inaccuracy; the two other behaviors have consistent accuracycomparing with the 97% accuracy obtained when walkingnaturally. Specifically, when the participant grabs the bag withone hand, the handbag is close to the participant’s lower limb.The signal reflections from the handbag can be captured withinthe radar’s FOV. Therefore, the step segments contain boththe handbag and user’s lower limbs, which confuses the CNNclassifier. We note that even with the interference from thehandbag, the lowest identification accuracy still achieves 87%.The results reveal that MU-ID effectively verifies the lowerlimb movement and is robust to various upper limb behaviors.Such robustness is essential to the practical deployment ineveryday life.

IX. CONCLUSION

In this paper, we study the feasibility of using mmWaveradar to perform multi-user identification, which could fa-cilitate various large-scale identity-based services. Towardsthis end, we design and implement MU-ID, a device-freegait-based multi-user identification system leveraging a singleCOTS mmWave radar. MU-ID exploits users’ spatial-temporallower limb motion features, including step length, duration,instantaneous lower limb velocity, and inter-lower limb dis-tance to construct user-specific gait models. To simultaneouslyidentify multiple users, we effectively separate the gait featuresfrom multiple users and perform silhouette-based analysis tosegment individual steps. A CNN-based classification methodis developed to extract gait features and perform multi-usergait identification. Extensive experiments involving ten par-ticipants show that MU-ID can achieve high identificationaccuracy and low false positive rate for both single andmultiple people scenarios. In addition, a comprehensive studyof different impacts (i.e., train/test split ratio, different en-vironments, and upper limb behaviors) further confirms therobustness of our system.

X. ACKNOWLEDGMENT

This work is supported in part by the National ScienceFoundation Grants CNS1820624, CCF1909963, CNS1717356,CNS1815908, and Army Research Office Grant W911NF-18-1-0221.

Page 10: MU-ID: Multi-user Identification Through Gaits Using ...

REFERENCES

[1] F. Adib, C.-Y. Hsu, H. Mao, D. Katabi, and F. Durand. Capturing thehuman figure through a wall. ACM Transactions on Graphics (TOG),34(6):219, 2015.

[2] F. Adib, Z. Kabelac, and D. Katabi. Multi-person localization via {RF}body reflections. In 12th {USENIX} Symposium on Networked SystemsDesign and Implementation ({NSDI} 15), pages 279–292, 2015.

[3] A. Ahmad, J. C. Roh, D. Wang, and A. Dubey. Vital signs monitoringof multiple people using a fmcw millimeter-wave sensor. In 2018 IEEERadar Conference (RadarConf18), pages 1450–1455. IEEE, 2018.

[4] S. P. Banerjee and D. L. Woodard. Biometric authentication andidentification using keystroke dynamics: A survey. Journal of PatternRecognition Research, 7(1):116–139, 2012.

[5] T. Calinski and J. Harabasz. A dendrite method for cluster analysis.Communications in Statistics-theory and Methods, 3(1):1–27, 1974.

[6] V. C. Chen, F. Li, S.-S. Ho, and H. Wechsler. Micro-doppler effect inradar: phenomenon, model, and simulation study. IEEE Transactions onAerospace and electronic systems, 42(1):2–21, 2006.

[7] J. A. Hartigan and M. A. Wong. Algorithm as 136: A k-means clusteringalgorithm. Journal of the Royal Statistical Society. Series C (AppliedStatistics), 28(1):100–108, 1979.

[8] M. Hu, Y. Wang, Z. Zhang, D. Zhang, and J. J. Little. Incremental learn-ing for video-based gait recognition with lbp flow. IEEE transactionson cybernetics, 43(1):77–89, 2012.

[9] T. I. Incorporated. Awr1642 single-chip 76-ghz to81-ghz automotive radar sensor evaluation module.http://www.ti.com/tool/AWR1642BOOST, 2019 (accessed July 26,2019).

[10] T. I. Incorporated. Real-time data-capture adapter for radar sensing eval-uation module. http://www.ti.com/tool/DCA1000EVM, 2019 (accessedJuly 26, 2019).

[11] N. Knudde, B. Vandersmissen, K. Parashar, I. Couckuyt, A. Jalalvand,A. Bourdoux, W. De Neve, and T. Dhaene. Indoor tracking of multiplepersons with a 77 ghz mimo fmcw radar. In 2017 European RadarConference (EURAD), pages 61–64. IEEE, 2017.

[12] S. Z. Li, R. Chu, S. Liao, and L. Zhang. Illumination invariant facerecognition using near-infrared images. IEEE Transactions on patternanalysis and machine intelligence, 29(4):627–639, 2007.

[13] M. Mercuri, I. R. Lorato, Y.-H. Liu, F. Wieringa, C. Van Hoof, andT. Torfs. Vital-sign monitoring and spatial tracking of multiple peopleusing a contactless radar-based sensor. Nature Electronics, page 1, 2019.

[14] R. Min, J. Choi, G. Medioni, and J.-L. Dugelay. Real-time 3dface identification from a depth camera. In Proceedings of the 21stInternational Conference on Pattern Recognition (ICPR2012), pages1739–1742. IEEE, 2012.

[15] M. P. Murray. Gait as a total pattern of movement: Including abibliography on gait. American Journal of Physical Medicine &Rehabilitation, 46(1):290–333, 1967.

[16] M. P. Murray, A. B. Drought, and R. C. Kory. Walking patterns ofnormal men. JBJS, 46(2):335–360, 1964.

[17] M. Patel, P. J. Vandevord, H. Matthew, B. Wu, S. Desilva, and P. H.Wooley. Video-gait analysis of functional recovery of nerve repairedwith chitosan nerve guides. Tissue engineering, 12(11):3189–3199,2006.

[18] Z. Peng, C. Li, J.-M. Munoz-Ferreras, and R. Gomez-Garcıa. Anfmcw radar sensor for human gesture recognition in the presence ofmultiple targets. In 2017 First IEEE MTT-S International MicrowaveBio Conference (IMBIOC), pages 1–3. IEEE, 2017.

[19] G. Porter and G. Doran. An anatomical and photographic technique forforensic facial identification. Forensic science international, 114(2):97–105, 2000.

[20] S. Rao. Introduction to mmwave sensing: Fmcw radars. TexasInstruments (TI) mmWave Training Series, 2017.

[21] Y. Ren, Y. Chen, M. C. Chuah, and J. Yang. User verification leveraginggait recognition for smartphone enabled mobile healthcare systems.IEEE Transactions on Mobile Computing, 14(9):1961–1974, 2014.

[22] C. Shi, J. Liu, H. Liu, and Y. Chen. Smart user authenticationthrough actuation of daily activities leveraging wifi-enabled iot. InProceedings of the 18th ACM International Symposium on Mobile AdHoc Networking and Computing, page 5. ACM, 2017.

[23] A. G. Stove. Linear fmcw radar techniques. In IEE Proceedings F(Radar and Signal Processing), volume 139, pages 343–350. IET, 1992.

[24] B. Vandersmissen, N. Knudde, A. Jalalvand, I. Couckuyt, A. Bourdoux,W. De Neve, and T. Dhaene. Indoor person identification using alow-power fmcw radar. IEEE Transactions on Geoscience and RemoteSensing, 56(7):3941–3952, 2018.

[25] R. Vera-Rodriguez, J. S. Mason, J. Fierrez, and J. Ortega-Garcia.Comparative analysis and fusion of spatiotemporal information forfootstep recognition. IEEE transactions on pattern analysis and machineintelligence, 35(4):823–834, 2012.

[26] C. Wang, J. Zhang, L. Wang, J. Pu, and X. Yuan. Human identificationusing temporal information preserving gait template. IEEE Transactionson Pattern Analysis and Machine Intelligence, 34(11):2164–2176, 2011.

[27] W. Wang, A. X. Liu, and M. Shahzad. Gait recognition using wifisignals. In Proceedings of the 2016 ACM International Joint Conferenceon Pervasive and Ubiquitous Computing, pages 363–373. ACM, 2016.

[28] L. Yujiri, M. Shoucri, and P. Moffa. Passive millimeter wave imaging.IEEE microwave magazine, 4(3):39–50, 2003.

[29] Y. Zeng, P. H. Pathak, and P. Mohapatra. Wiwho: wifi-based personidentification in smart spaces. In Proceedings of the 15th InternationalConference on Information Processing in Sensor Networks, page 4.IEEE Press, 2016.

[30] J. Zhang, B. Wei, W. Hu, and S. S. Kanhere. Wifi-id: Human identifica-tion using wifi signal. In 2016 International Conference on DistributedComputing in Sensor Systems (DCOSS), pages 75–82. IEEE, 2016.

[31] Z. Zhang, Z. Tian, and M. Zhou. Latern: Dynamic continuous handgesture recognition using fmcw radar sensor. IEEE Sensors Journal,18(8):3278–3289, 2018.

[32] H. Zhao, R. Mayzus, S. Sun, M. Samimi, J. K. Schulz, Y. Azar, K. Wang,G. N. Wong, F. Gutierrez Jr, and T. S. Rappaport. 28 ghz millimeter wavecellular communication measurements for reflection and penetration lossin and around buildings in new york city. In ICC, pages 5163–5167,2013.