Did You See Bob?Human Localization using Mobile Phones

Ionut ConstandacheDuke University

Presented by: Di ZhouSlides modified from Nichole Stockman

The Issue

• Finding someone in a public place can be difficult Might not know their

location

Maps and floor plans are not always available

Might be unfamiliar with the area

Hypothetical Scenario

• Mobicom conference – in a big hotel• Alice wants to meet her colleague Bob…– Can walk around– Can ask “Did you see Bob?”• But these can take a long time and he may be moving

– Can call him• But he may be in a meeting already

• Best to have someone escort you…

HoHow?w?

Current Solutions

• GPS– Drains the battery (and not very precise)– Outdoor scheme

• WiFi/GSM schemes– Not enough accuracy than GPS– Require special infrastructure, RF transmitter or

war-driving

Escort• Guide a user to the vicinity of a desired person– mobile phone sensors– Opportunistic encounters– Client/server model

• Does not require: - Physical coordination– GPS/Wi-Fi– War-driving– Maps or floor plans

• Can be easily installed and provides localization service when GPS/WiFi not available

Outline

• Motivation Recap:– We need efficient and accurate software for

people localization: Escort• Outline– Basic Design of Escort– Challenges– Solutions– Experiment Specifics– Results– Future Work

Basic Design Escort consists of two parts:

• Navigation- Get directions to the person

• Visual Identification- End to end human localization

Part 1 - Navigation• Clients report “Movement trail” periodically

– A(t), B(t), C(t)– <displacement, direction, time>– Use accelerometer and compass measurements– Displacement = # of steps multiply step size– Direction read from compass

• Also report “Encounters”– T_AC, T_BC– <movement trace, encounter time>– Definition ->Audio signals in inaudible frequencies within 5m

Server builds a virtual graph Global view of users’

positions and pathsA(t))

C(t)C(t)

B(t)B(t)

T_T_AACC

T_T_BBCC

B(t))

C(t))

Challenges

1. Accelerometers & Compasses are noisy– Measured path error over time

2. No global reference frame– How to correct errors?

3. Even if correct user position, difficult to correct entire trail

- Yet is necessary! (routing)

4. Trail graph grows over time

Solutions - 11. Accelerometers & Compasses are noisy– Use: (step size) * (step count)

Displacement error using Displacement error using double integrationdouble integration

Signature from up and down Signature from up and down bounce bounce of human body while walkingof human body while walking

Solutions - 1

1. Accelerometers & Compasses are noisy– Use: (step size) * (step count)

Error with step count method (avg 4%)Error with step count method (avg 4%)

Take into account varying Take into account varying step sizestep size(Vary step size with error factor drawn from Gaussian (Vary step size with error factor drawn from Gaussian

distribution centereddistribution centered

on 0 and standard deviation 0.15on 0 and standard deviation 0.15m)m)

Solutions - 2

2. No global reference frame– Use a fixed beacon transmitter

• Beacon Transmitter– Location is origin of a virtual coordinate system– Location diffusion ( single point updates )– Drift cancellation ( path correction ) sol’n 3

Solutions - 3• Drift Cancellation

– Amortize the correction vector over time– Assume that user’s projected path

deviates from the true path linearly over time

Solid Line: actual pathDotted Line: user-computed trailDashed Line: corrected user trail

User encounters beacon at t_r1,and another beacon or recentlyupdated user at t_r2.

Solutions 4• Graph computation done by Server– Pruning heuristic -> eliminate duplicates– Floyd-Warshall algorithm -> shortest paths

Graph – 4 users, 10 minGraph – 4 users, 10 min After pruningAfter pruning After Floyd-Warshall algAfter Floyd-Warshall alg

Basic Design Escort consists of two parts:

1. Navigation2. Visual Identification

Help you identify the person if she is someone you have not met before (e.g. first-time meeting with a professional colleague at a conference)

Basic Design – Visual Identification Perhaps Alice has not met Bob before…

Picture of face not enough

In a trusted environment like Mobicom, can

Take many photos of user to generate a “fingerprint”

Alice takes photos of surroundings. Image processing is done to identify Bob from the photos

Totally works in theory! Currently only implemented

offline and requires user input…

Experiment Specifics

• Parking Lot Experiment– 4 users, 13 min, phones in hand,

40 routing exp’mts– Used parking spot lines &

markers for ground truth (GPS not fine-grained enough)

• Indoor Experiment– 2 users, 6 min, 10 routing exp’mts

Results – Parking Lot

• Instantaneous location error over time

Results – Parking Lot

Instantaneous location error < 10m: Intertial ------------------------ ~6% of

cases Beacon & Encounter ------- ~ 68% of

cases Drift Cancellation ----------- ~ 84% of

cases

Final destination distance error 8.2m on average

Results – Indoor

Final destination distance error was 7m on average

Instantaneous location error: Better overall (due to indoor structure and

user guesswork)

Intertial --------------------- (N/A because no GPS indoors)

Beacon & Encounter ------ ~ 85% of cases Drift Cancellation ---------- ~ 90% of cases

Results – Visual Identification

• Is this a good result?– 80% accurate with 8 people in surroundings

Future Work

• “Escort is not designed for energy efficiency” – Turn off sensors (except accelerometer) when user not

moving– Less audio signaling when many users or beacons nearby– Less frequent uploading of data to server

• Routing through physical obstacles– People are smart– Visual representation of path may help

• Long routing paths– Include true-

direction arrow to Bob’s location

Future Work

• Routing instructions under low location accuracy– If update too far in past, prompt user to approach beacon– Update path as more info becomes available

• Phone placement– Need advancement of phone

gyroscopes to infer orientation

• Behavior under heavy user load– Scalability needs to be explored but should improve

performance (more beacons/users = more updated info, etc)

Thank You

Any Questions?