The Free Safety Problem Using Gaze Estimation as a Meaningful Input to a Homing Task Albert Goldfain...
-
Upload
buck-barker -
Category
Documents
-
view
216 -
download
0
Transcript of The Free Safety Problem Using Gaze Estimation as a Meaningful Input to a Homing Task Albert Goldfain...
The Free Safety Problem The Free Safety Problem Using Gaze Estimation as a Meaningful Input to a Homing TaskUsing Gaze Estimation as a Meaningful Input to a Homing Task
Albert GoldfainAlbert Goldfain
CSE 668: Animate Vision PrinciplesCSE 668: Animate Vision Principles
Final Project PresentationFinal Project Presentation
Defining the problemDefining the problem
American Football TerminologyAmerican Football TerminologyA “A “quarterbackquarterback” is an offensive player who is responsible for ” is an offensive player who is responsible for throwing the ball to “receivers”.throwing the ball to “receivers”.A “A “receiverreceiver” is an offensive (moving target) player who runs in a ” is an offensive (moving target) player who runs in a predefined pattern (or “route”) known to the quarterback and the rest predefined pattern (or “route”) known to the quarterback and the rest of his team. The receiver attempts to catch the ball if the of his team. The receiver attempts to catch the ball if the quarterback decides to throw to him.quarterback decides to throw to him.The “The “free safetyfree safety” is a defensive player who is responsible for ” is a defensive player who is responsible for impeding the targeted receiver from catching the ball. The free impeding the targeted receiver from catching the ball. The free safety is called “free” because he is not assigned to any particular safety is called “free” because he is not assigned to any particular receiver, and must make the decision of who to guard during play. receiver, and must make the decision of who to guard during play. He is labeled “safety” because he is usually the last line of defense He is labeled “safety” because he is usually the last line of defense and stands 6 or 7 yards behind all of the other defenders.and stands 6 or 7 yards behind all of the other defenders.
The goal of this project is to explore the visual aspects of the free safety’s decision making process
as he selects a receiver to guard.
Existing Literature/ResearchExisting Literature/Research
Gaze / Pose EstimationGaze / Pose Estimation– Perception of head orientation.Perception of head orientation. [Wilson et al] [Wilson et al]– Motion Segmentation and Pose Recognition with Motion History Motion Segmentation and Pose Recognition with Motion History
GradientsGradients. [Bradski, Davis]. [Bradski, Davis]– Head pose estimation without manual initialization.Head pose estimation without manual initialization. [Fitzpatrick] [Fitzpatrick]– 3D Face pose estimation and tracking from a monocular camera.3D Face pose estimation and tracking from a monocular camera. [Ji, [Ji,
Hu]Hu]– Pose determination of human faces by using vanishing pointsPose determination of human faces by using vanishing points. [Wang, . [Wang,
Sung]Sung]– Many, many more.Many, many more.
Computational Vision in SportsComputational Vision in Sports– RoboCupRoboCup [www.robocup.org] [www.robocup.org]– Computers Watching FootballComputers Watching Football. [MIT Media Lab/Vision and Modelling . [MIT Media Lab/Vision and Modelling
Group]Group]
Initial ObservationsInitial Observations
The problem is too large without domain knowledge/constraints.The problem is too large without domain knowledge/constraints.
A robotic (embodied computational) free safety would have to act very A robotic (embodied computational) free safety would have to act very quickly…no time for recovery paradigm approach…a good application for quickly…no time for recovery paradigm approach…a good application for active/animate vision.active/animate vision.
From the free safety’s point of view, the task of guarding a receiver can be From the free safety’s point of view, the task of guarding a receiver can be done using only two dimensions (width and depth). This is why a coach done using only two dimensions (width and depth). This is why a coach can draw x’s and o’s on a chalkboard without confusing his players. can draw x’s and o’s on a chalkboard without confusing his players.
Domain ConstraintsDomain Constraints
Players wear uniformsPlayers wear uniforms
Markings on field can be used to determine distances with Markings on field can be used to determine distances with precisionprecision
Good lighting can be assumedGood lighting can be assumed
SidelinesSidelines
2D internal representation for free safety2D internal representation for free safety
QB head pose restrictionsQB head pose restrictions
Quarterback Centered Coordinate Quarterback Centered Coordinate SystemSystem
1
(0)(0)
n
i
is
xx
n
• At each instant t, wide receiver i’s location is given by the parametric coordinates (xi(t),yi(t)) in this space.
• The initial pre-snap values for the safety position are given by
(0)s losy D
Safety Centered Polar Coordinate Safety Centered Polar Coordinate SystemSystem
• Coordinates for wide receiver i can be expressed with a distance from the safety ri and rotation angle s
• We can convert from safety coordinates to quarterback coordinates using
( ) cos
( ) sin
i i si
i i si
x t r x
y t r y
Finding the QuarterbackFinding the QuarterbackSafety could use one of two techniques:
• Use constraints of football formations to determine who is behind the center (a lineman on the line of scrimmage) at time t0
• Search image for a player that has the ball immediately after time t0
Once the quarterback is found, the head pose image must be segmented out.
Given a new image…
…find best correlation to a stored image
Image Database: Views of Quarterback Pose at 10 deg. Angle Increments.
Using Image Difference Matching Using Image Difference Matching as a Similarity Metricas a Similarity Metric
• Difference will be minimized when test image and stored image are the most similar. Thus the darkest of these image differences should be closest match.• The catch: often times, symmetries in image differences at certain angles will yield false positives. Perhaps image difference is a weak similarity metric!• The solution: Look at the angular “nearest neighbors” of a proposed solution.
Some Test Results for Image Some Test Results for Image Difference ClassificationDifference Classification
Test Head Pose 1
0
20
40
60
80
100
1 3 5 7 9 11 13 15 17 19
Stored Image Index
Ab
so
lute
Im
ag
e
Dif
fere
nce
Test Head Pose 2
0
20
40
60
80
100
1 3 5 7 9 11 13 15 17 19
Stored Image Index
Ab
solu
te Im
age
Dif
fere
nce
Acute Angle Obtuse Angle
Test Image Stored Image Test Image Stored Image
Labeling the Receivers in Labeling the Receivers in Quarterback’s FOVQuarterback’s FOV
Safety begins turning motion based on Safety begins turning motion based on θθqq
If no receiver is found, safety turns back to the If no receiver is found, safety turns back to the quarterback to estimate quarterback to estimate θθqq once again. once again.
If multiple receivers are found, a suggested If multiple receivers are found, a suggested egomotion “action” is output and egomotion “action” is output and θθqq is estimated is estimated again.again.If only one receiver is found, the safety approaches If only one receiver is found, the safety approaches that receiver.that receiver.
Further Work / Potential Further Work / Potential ApplicationsApplications
Working on decision making egomotion “action” function Working on decision making egomotion “action” function for the safety. Such a function needs to balance the for the safety. Such a function needs to balance the early goal of maintaining a large field of view with the early goal of maintaining a large field of view with the later goal of approaching the wide receivers.later goal of approaching the wide receivers.
This function needs to be time based. If the defensive This function needs to be time based. If the defensive team is doing its job, the quarterback should be running team is doing its job, the quarterback should be running out of time to throw. out of time to throw.
I am willing to take any suggestions I am willing to take any suggestions I believe further research of this problem could yield I believe further research of this problem could yield important results in robotics, video game AI and important results in robotics, video game AI and computer vision in general.computer vision in general.