PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted...
Transcript of PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted...
![Page 1: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/1.jpg)
PoDC: WiFi spyingSeeing, keylogging and hearing through walls
François Wirz | 23-05-2017
papers: See Through Walls with WiFi!Tracking Keystrokes Using Wireless SignalsWe Can Hear You with Wi-Fi!
![Page 2: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/2.jpg)
Motivation: Sensing with Wireless Signals
● Coarse granularity○ tracking position in room
2image sources: all images are adapted from the respective paper and poster, except where stated otherwise
![Page 3: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/3.jpg)
Motivation: Sensing with Wireless Signals
● Coarse granularity○ tracking position in room
● Fine granularity○ tracking gesture through walls○ binary H2M communication through walls
3
![Page 4: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/4.jpg)
Motivation: Sensing with Wireless Signals
● Coarse granularity○ tracking position in room
● Fine granularity○ tracking gesture through walls○ binary communication through walls
● Very fine granularity○ tracking keystrokes○ tracking lip movements
4
![Page 5: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/5.jpg)
Technical Primer
Constructive interference Destructive interference
image src: http://pediaa.com/difference-between-constructive-and-destructive-interference/image sources: all images are adapted from the respective paper, except where stated otherwise
5
amplitude amplitude
time time
![Page 6: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/6.jpg)
Technical Primer: Interference Nulling
Interference nulling with two sourcesimage src: https://people.csail.mit.edu/fadel/papers/wivi-poster.pdfimage src: http://pediaa.com/difference-between-constructive-and-destructive-interference/
6
![Page 7: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/7.jpg)
Technical Primer: Beamforming
Beamforming through constructive interferenceimage src: https://en.wikipedia.org/wiki/Phased_array
7
![Page 8: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/8.jpg)
Technical Primer: Multiple-Input Multiple-Output
● MIMO is used in:○ WiFi 802.11n standard
○ LTE standard
○ Power-line communication
image src: http://www.fastweb.it/var/storage_feeds/CMS/articoli/f77/f77aa70d79b114d8f2c9ea49eb5719ba/mimio-graphic_0.jpg
8
![Page 9: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/9.jpg)
Technical Primer: Multiple-Input Multiple-Output
● MIMO○ allows to focus the signal emitted (beamforming)
○ allows signal to cancel out in a plane (interference nulling)
○ can use multiple senders or multiple receivers or both
○ more uniform signal that can be amplified (no receiver saturation)
image src: http://www.tomsitpro.com/articles/wi-fi-performance-problems,2-238.html
9
![Page 10: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/10.jpg)
Technical Primer: Flash effect
● Flash effect○ most of the signal gets reflected by the first obstacle○ cancels out all weaker signal from behind
■ signal from bodies is drowned in noise■ cannot amplify signal because receiver would saturate
image src: https://people.csail.mit.edu/fadel/papers/wivi-poster.pdf
10
![Page 11: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/11.jpg)
Technical Primer: Flash effect○ Other approaches use larger devices:
■ 2 GHz of bandwidth (UWB)
■ strong power source
■ large antenna array (2.5 m)
image src: https://www.researchgate.net/.../publication/292148851....pdf, bitly: http://bit.ly/2qnnwvD
11
![Page 12: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/12.jpg)
Technical Primer: Inverse Synthetic Aperture
● Synthetic Aperture Sensing
image src: https://people.csail.mit.edu/fadel/papers/wivi-poster.pdf
12
● Inverse Synthetic Aperture Sensing○ use temporal signal to extract spatial information○ obtain angle of motion
![Page 13: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/13.jpg)
See Through Wall: WiVi
● Applying these techniques with WiVi:
13
WiVi
![Page 14: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/14.jpg)
See Through Wall: WiVi
14
WiVi
video src: https://youtu.be/uJkQzLjYBFI?t=6
![Page 15: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/15.jpg)
WiVi: angle and motion
15
WiVi
![Page 16: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/16.jpg)
WiVi: Gesture encoding
● Standard Return-to-zero encoding○ Encode 0 bit as step forward, step back○ Encode 1 bit as step back, step forward
16
WiVi
![Page 17: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/17.jpg)
● Property used○ MIMO interference nulling at wall, first obstacle○ Inverse Synthetic Aperture for emulated antenna array
See Through Wall: WiVi
image src: https://people.csail.mit.edu/fadel/papers/wivi-poster.pdf
17
WiVi
![Page 18: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/18.jpg)
● Property used○ MIMO interference nulling at wall, first obstacle○ Inverse Synthetic Aperture for emulated antenna array
● Objective achieved○ Overcome flash effect○ Have a portable solution
See Through Wall: WiVi
image src: https://people.csail.mit.edu/fadel/papers/wivi-poster.pdf
18
WiVi
![Page 19: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/19.jpg)
Tracking Keystrokes using Wireless Signals
image src: https://c1.staticflickr.com/6/5053/5498834683_f579445346_b.jpg
19
WiKeylog
![Page 20: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/20.jpg)
Tracking Keystrokes using Wireless Signals
20
WiKeylog
![Page 21: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/21.jpg)
Tracking Keystrokes using Wireless Signals
21
WiKeylog
![Page 22: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/22.jpg)
From phase to delay
● Get delay introduced by keystroke by converting phase shift into delay● Capture delay effect by using cancellation at receiver
22
WiKeylog
![Page 23: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/23.jpg)
From delay to keystroke
● Measure trough location to infer change in channel● Introduce artificial delay to make trough more significant
23
Pow
er le
vel
Frequency
WiKeylog
![Page 24: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/24.jpg)
The keystroke tracking system
24
WiKeylog
![Page 25: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/25.jpg)
Tracking Keystrokes: Performance
25
WiKeylog
Repeated key 5 on keypad: accuracy
![Page 26: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/26.jpg)
26
WiKeylog
Full key range, partially trained
Tracking Keystrokes: Performance
![Page 27: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/27.jpg)
● Property used○ Shift in frequency of cancellation through caused by phase shift of channel○ Finger modeled as source of multipath signal
Tracking Keystrokes using Wireless Signals
27
WiKeylog
![Page 28: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/28.jpg)
● Property used○ Shift in frequency of cancellation through caused by phase shift of channel○ Finger modeled as source of multipath signal
● Objective achieved○ first passive, single receiver keystrokes tracking system○ agnostic of physical layer and MAC protocols
Tracking Keystrokes using Wireless Signals
28
WiKeylog
![Page 29: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/29.jpg)
We Can Hear You with WiFi: WiHear
Device free, non-invasive remote ‘hearing’ 29
![Page 30: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/30.jpg)
We Can Hear You with WiFi
30
![Page 31: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/31.jpg)
Mouth motion profiling
● Locating mouth
31
![Page 32: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/32.jpg)
Mouth motion profiling
● Filtering out-band interferences○ cancel high frequency interferences○ remove both static interferences and winking using band-pass filter (red boxes)
32
![Page 33: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/33.jpg)
Mouth motion profiling
● Partial multipath removal○ Convert Channel State Information to time domain via IFFT
33image src: https://upload.wikimedia.org/wikipedia/commons/6/64/FFT_of_Cosine_Summation_Function.png
![Page 34: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/34.jpg)
Mouth motion profiling
● Partial multipath removal○ Convert Channel State Information to time domain via IFFT○ Remove multipath >500 ns
34image src: https://dl.acm.org/citation.cfm?doid=2462456.2464463
![Page 35: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/35.jpg)
Mouth motion profiling
● Partial multipath removal○ Convert Channel State Information to time domain via IFFT○ Remove multipath >500 ns○ Convert CSI back to frequency domain via FFT
35image src: https://upload.wikimedia.org/wikipedia/commons/6/64/FFT_of_Cosine_Summation_Function.png
![Page 36: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/36.jpg)
Mouth motion profiling
● Partial multipath removal○ Convert Channel State Information to time domain via IFFT○ Remove multipath >500 ns○ Convert CSI back to frequency domain via FFT
● Rational○ mouth motion is non-rigid compared to other body movements○ multipath reflections with similar delays do all contain information about the mouth motion
36image src: https://dl.acm.org/citation.cfm?doid=2462456.2464463
![Page 37: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/37.jpg)
Mouth motion profiling
● Apply discrete wavelet packet decomposition○ easier signal analysis on time and frequency domain○ allows multi-scale analysis
37
![Page 38: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/38.jpg)
Learning based lip reading
38
![Page 39: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/39.jpg)
Learning based lip reading
● Word segmentation● Inner-word segmentation● Feature extraction● Classification
39
![Page 40: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/40.jpg)
We Can Hear You with WiFi
40
![Page 41: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/41.jpg)
● Property used○ MIMO beamforming, focused on mouth○ Partial multipath effect, partially remove multipath after wavelet packet transformation
We Can Hear You with WiFi
41
![Page 42: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/42.jpg)
● Property used○ MIMO beamforming, focused on mouth○ Partial multipath effect, partially remove multipath after wavelet packet transformation
● Objective achieved○ lip reading and speech recognition without line of sight○ Context aware speech recognition enhancement
We Can Hear You with WiFi
42
![Page 43: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/43.jpg)
Conclusion
● All three very innovative● Early stage proofs of concept● Novel use cases requiring NLOS sensing● Far reaching privacy implications● The ISM band can be used for more than machine to machine
communication, e.g. indoor localization, sensing and control
43
WiKeylogWiVi
![Page 44: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/44.jpg)
Follow up results
44video src: https://youtu.be/sbFZPPC7REc?t=122
![Page 45: PoDC: WiFi spying · WiFi 802.11n standard LTE standard ... MIMO allows to focus the signal emitted (beamforming) allows signal to cancel out in a plane (interference nulling) can](https://reader033.fdocuments.in/reader033/viewer/2022060522/60513137a5bc231ff421df17/html5/thumbnails/45.jpg)
Follow up results
45video src: https://youtu.be/3Atky2Jt_-4?t=3