Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an...
-
Upload
julie-magdalen-stewart -
Category
Documents
-
view
213 -
download
1
Transcript of Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an...
![Page 1: Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an interpretation about its environment. Groups of robots can combine.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c28140/html5/thumbnails/1.jpg)
Robot Vision:Multi-sensor Reconnaissance
![Page 2: Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an interpretation about its environment. Groups of robots can combine.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c28140/html5/thumbnails/2.jpg)
Overview
• An individual robot can develop an interpretation about its environment.
• Groups of robots can combine their resources to develop an improved, collective interpretation.
• Such a “swarm” is reliant on centralized processing to refine the collective interpretation.
• The swarm has more insight than solitary individuals.
![Page 3: Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an interpretation about its environment. Groups of robots can combine.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c28140/html5/thumbnails/3.jpg)
Single-sensor Vision
• Our setup involves numerous robots, each with a single video camera.
• What can a single-camera robot see?
• A background image may be used to establish a baseline of irrelevant imagery.
• Anything that differs from the background is considered relevant – background subtraction.
minus equals
![Page 4: Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an interpretation about its environment. Groups of robots can combine.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c28140/html5/thumbnails/4.jpg)
Making Sense of Pixel Data• Background subtraction and classical image processing algorithms can
mark every pixel in every frame as foreground (interesting) or background (uninteresting).
• Conventional techniques can clean noise.
• Regions of contiguous foreground pixels are likely to constitute a single object: a “blob.”
• This assumption is false when one object occludes another.• Later we will see a way around this.
![Page 5: Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an interpretation about its environment. Groups of robots can combine.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c28140/html5/thumbnails/5.jpg)
Developing the Individual’s Interpretation
• Pixel-space blobs have measurable properties: color, shape, and location.
• Geometric properties (location, size) are relative to the camera’s perspective.
• A single robot can provide a list of objects with a bearing to each, but has no depth perception.
• Each robot’s interpretation may be combined to form a much stronger interpretation for the entire swarm.
![Page 6: Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an interpretation about its environment. Groups of robots can combine.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c28140/html5/thumbnails/6.jpg)
Developing the Swarm’s Interpretation
• Each robot has a list of bearings to potential objects. This information can be visualized as rays originating from the robot’s location.
• The intersections of these rays represent potential objects in 3D space.
• Many intersections are bogus.
• Many intersections conflict with others – each ray can only correspond to one object.
![Page 7: Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an interpretation about its environment. Groups of robots can combine.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c28140/html5/thumbnails/7.jpg)
Culling False Objects• Algorithm: group compatible locations together. This yields disjoint
sets of intersections that can coexist.
• The set with the most supporting evidence wins.
![Page 8: Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an interpretation about its environment. Groups of robots can combine.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c28140/html5/thumbnails/8.jpg)
Combining the Individual and Swarm Interpretations
• At this point we have a set of objects with 3D locations.
• Individual robots can provide silhouettes of the objects.
• This information may be combined to create a 3D shape.
• Incorporating past history can strengthen our conclusions.
![Page 9: Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an interpretation about its environment. Groups of robots can combine.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c28140/html5/thumbnails/9.jpg)
3D Hulls• Each camera contributes a silhouette of an object, and a
ray on which the silhouette lies.• Projecting the silhouette along the ray forms a “cone”.• The intersection of these cones carves a 3D solid; imagine
pushing cookie cutters through space.• The solid is guaranteed to enclose the true shape, but will
be convex, i.e., ignores indentations. • Such an upper bound is termed a hull.• Hulls are typically represented as a mesh of triangles.
![Page 10: Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an interpretation about its environment. Groups of robots can combine.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c28140/html5/thumbnails/10.jpg)
Applications of 3D Hulls
• The 2D silhouettes can be “painted” on the mesh.
• The solid can be rendered from any angle.
• 3D shape may be used to classify objects – threat assessment, for example.
• Meshes may be recorded for future use…
![Page 11: Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an interpretation about its environment. Groups of robots can combine.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c28140/html5/thumbnails/11.jpg)
Establishing Object Tracks
• Object records from successive frames can be combined to establish a log of known objects.
• These “object tracks” can aid future processing, establishing a positive feedback loop:
– Distinguishing between one large object, and two objects close to one another.
– Past motion can predict where objects will be located, minimizing the occlusion problem mentioned earlier.
– Past hulls can predict how an object’s silhouette will appear in each camera.
![Page 12: Robot Vision: Multi-sensor Reconnaissance. Overview An individual robot can develop an interpretation about its environment. Groups of robots can combine.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c28140/html5/thumbnails/12.jpg)
Summary
• Swarms of camera-equipped robots can collaborate to track and model objects in space.
• The swarm’s results are more concrete than any individual’s observations.
• Observation is passive and uses relatively few resources (weight, energy, money).