SixthSense:RFID-based Enterprise
Intelligence
Lenin Ravindranath, Venkat Padmanabhan (MSR India)
Piyush Agrawal (IIT Kanpur)
RFID Radio Frequency Identification Components
RFID Reader with Antennas Tags (Active and Passive)
Electromagnetic waves induce current Tag responds
Globally unique ID Data
UHF (865-956 MHz) Range – up to 7m
Applications Tracking, Inventory, Supply Chain, Authentication, … Novel Research Applications
Motivating Scenario
Lenin missed an object in the
conference room – 2nd floor Scientia
SixthSense Goal Making people, objects and workspaces, the first class
citizens of the enterprise computing
Components Use RFID to capture the rich interaction between
people and their surroundings Combine with other enterprise systems/sensors to
make automated inferences Enable Useful Services
Setting
People and objects tagged
Camera
Calendar
Presence
RFID Antennas
Assumptions Widespread coverage of RFID readers in the workspace Users are free to pick up new tags and affix them to
objects Can put multiple tags on an object
No dependence on cataloging Cataloging is an overhead
TagID Entity Antenna Workspace Error Prone Tags are fragile – may have to be replaced Readers/Antenna could be moved
Start with an undifferentiated mass of tags and infer everything
Lenin missed an object in the
conference room – 2nd floor Scientia
13548234 – Ant 115574523 – Ant 113548234 – Ant 615574523 – Ant 1
Architecture
People and objects tagged
Camera
Calendar
Presence
RFID Antennas
SixthSense
SixthSense
SixthSense
Automated Inference Platform
Programming Model Applications
Inference Engine Person-Object Differentiation Object Ownership Inference Zone Identification Person Identification Person-Object Interaction
Person-Object Differentiation
People can move on their own Objects move only when carried by a person
Co-movement based heuristic Relative Movement (RM)
Zone 1 Zone 2 Zone 3
Object Ownership Inference Co-Presence
Calculate the amount of time the object is concurrently present in the same zone as a person
Owner is the person with which the object is co-present the most and greater than a threshold
Person Identification
1
xyz abc
1
2
Workspace Entrance
Event
Log-in event
Coincidence count
1
xyz
1
time
t1
t2
1
Object Interaction (only in zones of interest) Intra zone Identify interaction in zones of interest
A person lifted an object A person turned the orientation an object
Signal Strength of tag varies Change in distance Change in orientation Contact
Monitor variation in RSSI
Object Interaction Sample the RSSI of each object tag every 200ms Sliding 4-second wide window
Difference between the 10th percentile and 90th percentile of the RSSI
Object is said to be interacted - If the difference > threshold
Minimizing spurious detections Use multiple antennas
Object Interaction
Antenna 1
Antenna 2
Interacted
Interacted
Ensuring Privacy Enterprise will deploy and manage the system
Expose appropriate set of information Trust - Analogous to the enterprise e-mail system
Defend against rogue readers Relabeling approach [A. Juels, 2006]
EPC code rewritten at random times SixthSense will be aware of the mapping
between the old and new tag IDs
Implementation
Simulator(Trace Generation)
Experimental Setup(Real-time feed)
Experimental Setup
Results Inter-zone movement detection
Object Reliability (1m) Reliability (2m)
Badge on belt clip 100% 96%
Small box in hand 94% 88%
Object Interaction
Testbed deployment To make correct inferences
Average inter-zone movements needed – 4 Average log-ins required - 3
Distance between antennas
Detection time
1.5m 2.39s
2m 3.4s
2.5m 5.03s
Simulation Probabilistic model to generate artificial traces Simulated
Inter-zone movement (walk) People carrying multiple objects Log-in events Untagged people
Zone 1 Zone 2 Zone 3
Results Person-object differentiation Person Identification Varying average walk length Effects of untagged people
Person-Object differentiation and ownership
20 people, 100 tags, probability of walk – 0.1, walk length - 5
Person Identification
10% of users entering workspace simultaneously
Programming Model Callbacks
InterZoneMovementEvent (tagID, startZone, endZone, Time)
ObjectInteractedEvent(tadID, Zone, Time)
Lookups GetTagList() GetPersonTags() GetOwnedObjects(tagID) GetTagType(tagID) GetTagOwner(tagID) GetPersonTagIdentity(tagID) GetZoneType(zone) GetTagsInZone(zone) GetTagWorkspaceZone(tagID) GetCurrentTagZone(tagID) GetCalendarEntry(ID, Time)
Example Misplaced Object Alert
personTags = GetPersonTags()
For each ownerTagID in personTags
ObjTags = GetOwnedObject(ownerTagID)
OwnerZone = GetCurrentTagZone(ownerTagId)
OwnerWorkspace = GetTagWorkspaceZone(ownerTagId)
For each obj in ObjTags
objZone = GetCurrentTagZone(ownerTagId)
if (objZone != OwnerZone && objZone != OwnerWorkspace)
Raise Alert
Applications Annotated video Semi-automated image catalog Misplaced object alert Automatic conference room booking
Annotating video with physical events Events
Inter-zone movements Object Interaction
Tag video feed with events Person X interacted an object Y
Rich video database Support rich queries
Give me all videos where Person A interacted with Object B
Application: Integrate with enterprise security camera system
Semi-automated Image catalog TagIDs are not user friendly Catalog tagID with its Image
User picks up an object Shows before the camera and takes a photo Automatic cataloging (TagID, Image)
Annotating video with physical events
Related Work Localization
LANDMARC Indoor Location Sensing Using Active RFID
Sherlock (UMass) Automatically locating objects for humans
Ferret (UMass) RFID Localization for Pervasive Multimedia
Platform Cascadia (UWashington)
Specifying, detecting and managing RFID events
Object Interaction I sense a disturbance in the force (Intel Research, Seattle)
Unobtrusive detection of Interactions with RFID-tagged Objects
With other sensors Fusion of RFID and Computer Vision (MSR)
Summary SixthSense
Enterprise Setting People and Objects tagged RFID with other enterprise sensors
Components Automated Inference Platform Applications
http://research.microsoft.com/research/mns/projects/SixthSense/
Questions?
Backup
Semi-Automated Image Catalog TagID-Image Cataloging
User picks up a tagged object Hold it in front of the camera Clicks a picture Automatically identify the tagID of the object
SixthSense SystemInference Engine, Database, Applications
RFID Reader
RFID Antennas
Calendar Data Presence Data
ApplicationsQueries
Industry Tracking, Inventory, Supply Chain, Authentication
Research Measurements Improving reliability, security Localization RFID + Computer Vision Interaction detection RFID + other sensors
RFID Applications
Person-Object Differentiation People can move on their own Objects move only when carried by a person
Co-movement based heuristic For every tag T, find co-movement tag set {T1, T2..Tn}
m – total inter-zone movement of T mi – total inter-zone movement of Ti
ci – amount of co-movement exhibited by Ti with T
Declare the tag with the highest RM as person Eliminate this tags movements Repeat the algorithm till RM is positive Tags with negative RM are objects
Zone Identification Individual workspace
If there is one person predominantly present in a zone Workspace of that person
Shared workspace If no one person is predominantly present in a zone Length of time from a person entry to exit < threshold
Reserved shared workspace Length of time people are present > threshold Common meeting entries in their calendars
Common areas Any space that is not classified as one of the above
Challenges – Improving Reliability Multi-tagging scheme
Affix multiple tags in different orientation Increases the probability that atleast one of the tags being
detected
Automatic Inference Initially assume all tags belong to one giant super object
Fully connected graph When two tags are detected simultaneously in different
zones Tags belong to different objects Delete edges between them
Connected components Set of tags attached to the same object
Evaluation – with untagged people
Automatic Conference Room Booking Conference Room Zone is automatically
identified Reserved Space
Automatically book conference room If it is not reserved And bunch of people go into the conference room And spend say 5 minutes
Discussion Privacy
Deployed and managed by enterprise Limited access to users Relabeling approach
Economic Feasibility Passive Tags are cheap Prices are RFID readers expected to drop (Intel
R1000) Health Implications
Transmitted RF power (up to 2W) is well within safe limits
this question will undoubtedly continue to receive much attention and study
Top Related