Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar...
-
Upload
rosamund-jacobs -
Category
Documents
-
view
225 -
download
0
Transcript of Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar...
![Page 1: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/1.jpg)
Intel LabsIntel Labs
Self Localizing sensors Self Localizing sensors and actuators on and actuators on Distributed Computing Distributed Computing PlatformsPlatforms
Vikas RaykarVikas Raykar
Igor KozintsevIgor Kozintsev
Rainer LienhartRainer Lienhart
![Page 2: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/2.jpg)
MotivationMotivation Many multimedia applications are emerging which use multiple audio/video sensors and actuators.
Microphones
Cameras
Speakers
Displays
Dis
trib
ute
d
Cap
ture
Dis
trib
ute
d
Ren
der
ing
Other Applications
Number Crunching
![Page 3: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/3.jpg)
ApplicationsApplications
Audio/Video Surveillance
Hands free voice communication
MultiChannel Speech Enhancement
Smart ConferenceRooms
Audio/Image Based Rendering
Object LocalizationAnd tracking
Meeting Recording
Distributed AudioVideo Capture
Interactive Audio Visual Interfaces
MultiChannel EchoCancellation
Speech Recognition
Source separation andDeverberation
![Page 4: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/4.jpg)
Additional MotivationAdditional Motivation Current work has focused on setting up all the sensors and actuators on a single dedicated computing platform.
Dedicated infrastructure required in terms of the sensors, multi-channel interface cards and computing power.
Computing devices such as laptops, PDAs, tablets, cellular phones, camcorders have become pervasive.
Audio/video sensors on different laptops can be used to form a distributed network of sensors.
On the other hand…
![Page 5: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/5.jpg)
Problem formulationProblem formulationPut all the distributed audio-visual I/O capabilities into a common time and space.
In this paper:Focus on providing a common space by means of actively estimating the 3D positions of the sensors (microphones) and actuators (speakers).
Account for the errors due to lack of temporal synchronization among various sensors and actuators (A/Ds and D/As) on distributed general purpose computing platforms.
![Page 6: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/6.jpg)
Our View of Distributed Our View of Distributed Sensor NetworkSensor Network
X
Y
Z
![Page 7: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/7.jpg)
Localization with known Localization with known positions of speakerspositions of speakers
Distances are not exact
There are more speakers
![Page 8: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/8.jpg)
If positions of speakers are If positions of speakers are unknown…unknown…
Consider M Microphones and S speakers.What can we measure?
Distance between each speaker and all microphones (Time Of Flight)
MxS TOF matrix
Assume TOF corrupted by AWGN: can derive the ML estimate.
Calibration signal
![Page 9: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/9.jpg)
Nonlinear Least SquaresNonlinear Least Squares
Find the coordinates which minimizes this
![Page 10: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/10.jpg)
Reference Coordinate SystemReference Coordinate System
X axis
Positive Y axis
OriginSimilarly in 3D
1.Fix origin (0,0,0)
2.Fix X axis
(x1,0,0)
3.Fix Y axis
(x2,y2,0)
4.Fix positive Z axis
x1,x2,y2>0
Which to choose? Later…
![Page 11: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/11.jpg)
Intel LabsIntel Labs
On a synchronized platform all is On a synchronized platform all is well..well..
![Page 12: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/12.jpg)
However on a Distributed However on a Distributed system..system..
![Page 13: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/13.jpg)
Intel LabsIntel Labs
PC platform overviewPC platform overview
PCI SlotsPCI Slots
CPUCPU
AG
PA
GP MCH
ICH
ATAATA
LAN LAN
USBUSB
AC97AC97ICH, hub,
PCI, LAN, etc.
CPU, MCH, FSB, memory
Operating system
Multimedia/multistream applications
Audio/video I/O devices
I/O bus
![Page 14: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/14.jpg)
t
t
jtsSignal Emitted by source j
Signal Received by microphone i
ijFOT ˆ
itmijTOF
Capture Started
Playback Started
Time Origin
Timing on distributed systemTiming on distributed system
![Page 15: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/15.jpg)
Speaker Emission Start Times
S
Microphone Capture Start Times
M -1Assume tm_1=0
Microphone and speakerCoordinates
DM+DS - [ D(D+1)/2 ]
MS TOF Measurements
Joint EstimationJoint Estimation
![Page 16: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/16.jpg)
Formulation same as above but less number of parameters.
Time Difference of Arrival (TDOA)Time Difference of Arrival (TDOA)
![Page 17: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/17.jpg)
Levenberg Marquadrat method
Multidimensional function.
Unless we have a good initial guess may not convergeto the global minima.
Approximate initial guess required.
Nonlinear least squaresNonlinear least squares
![Page 18: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/18.jpg)
dot product matrixSymmetric positive definiterank 3
Given B can you get X ?....Singular Value Decomposition
Multi Dimensional ScalingMulti Dimensional Scaling
![Page 19: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/19.jpg)
Clustering approximationClustering approximation
![Page 20: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/20.jpg)
i i
j i
j j
i j
Clustering approximationClustering approximation
![Page 21: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/21.jpg)
k
ijd
kjd
kid
i
j
How to get dot product from the How to get dot product from the pair wise distance matrixpair wise distance matrix
![Page 22: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/22.jpg)
Later shift it to our
orignal reference
Slightly perturb each location of GPCinto two to get the initial guess for the microphone and speaker coordinates
Centroid as the originCentroid as the origin
![Page 23: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/23.jpg)
Sample result in 2DSample result in 2D
![Page 24: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/24.jpg)
ApproxDistance matrixbetween GPCs
Approxts
Approx tm
Clustering
Dot product matrix
Dimension and coordinate system
MDS to get approx GPC locations
perturb
TOF matrix
Approx. microphone and speaker
locations
TDOA basedNonlinear
minimization
Microphone and speakerlocations tm
AlgorithmAlgorithm
![Page 25: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/25.jpg)
Gives the lower bound on the variance of any unbiased estimator.
Does not depends on the estimator. Just the data and the noise model.
Basically tells us to what extent the noise limits our performance i.e. you cannot get a variance lesser than the CR bound.
Jacobian
Rank deficit: remove theknown parameters
Cramer-Rao boundCramer-Rao bound
![Page 26: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/26.jpg)
Performance comparisonPerformance comparison
![Page 27: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/27.jpg)
Dependence on number of nodesDependence on number of nodes
![Page 28: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/28.jpg)
Dependence on number of nodesDependence on number of nodes
![Page 29: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/29.jpg)
Geometry mattersGeometry matters
![Page 30: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/30.jpg)
Geometry mattersGeometry matters
![Page 31: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/31.jpg)
Mic 3
Mic 1
Mic 2
Mic 4
Speaker 1
Sp
eake
r 4S
pea
ker
2
Speaker 3
X
Z
Roo
m L
engt
h =
4.2
2 m
Room Width = 2.55 m
Room Height = 2.03 m
Experimental setup: bias 0.08 cm Experimental setup: bias 0.08 cm sigma 3.8 cmsigma 3.8 cm
![Page 32: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/32.jpg)
Intel LabsIntel Labs
SummarySummary General purpose computers can be used for General purpose computers can be used for
distributed array processingdistributed array processing It is possible to define common time and space for a It is possible to define common time and space for a
network of distributed sensors and actuators.network of distributed sensors and actuators. For more information please see our two papers in For more information please see our two papers in
ACM MM in November or contact ACM MM in November or contact [email protected] [email protected] [email protected]@intel.com
Let us know if you will be interested in testing/using Let us know if you will be interested in testing/using out time and space synchronization software for out time and space synchronization software for developing distributed algorithms on GPC (available developing distributed algorithms on GPC (available in November)in November)
![Page 33: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/33.jpg)
Intel LabsIntel Labs
BackupBackup
![Page 34: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/34.jpg)
Calibration signalCalibration signal
![Page 35: Intel Labs Self Localizing sensors and actuators on Distributed Computing Platforms Vikas Raykar Igor Kozintsev Igor Kozintsev Rainer Lienhart.](https://reader036.fdocuments.in/reader036/viewer/2022062423/56649ea25503460f94ba6953/html5/thumbnails/35.jpg)
Results (contd.)Results (contd.)