PowerPoint Presentation

Software design innovation:to catch a ballJoseph martin

Hello, my name is Joseph Martin. I am a senior on the Gorham and Falmouth robotics team, and this year our team has developed an innovative method for catching the ball.1It takes a team

Building a robot requires skills across many disciplines6 weeks isnt very much timeCreating a robot requires lots of experience; more experienced members must share their experience with newer members

Building a robot from scratch is the epitome of multidisciplinary. Almost no one possesses all the necessary skills to create a robot; it takes a team with a broad distribution of skills. With only six weeks, teamwork is even more important, as it becomes necessary to do as much as possible at the same time: while the design sub-team is finalizing the design, the programming sub-team can be working out kinks in the drivetrain code, and the electrical sub-team can be creating a neat electrical board. Since we lose experienced members and gain newer, less experienced members each year, teaching skills and cooperation become invaluable in achieving one of the main goals of FIRST: inspiring students and giving them the necessary skills to become leaders in science and technology. Even though I can count on one hand the number of times Ive picked up a drill, the team counts on me for innovative software design.2The vision co-processors

USB ports for USB camerasConvenient 5v Power ConnectionsEthernet ports for communication with cRIOCustom case to easily mount both BeagleBones2x BeagleBone Black, each with a 1GHz ARM Cortex A8 CPU

Our robot may look like many others, but we have a secret weapon: not one, but two vision coprocessors. Each coprocessor is a BeagleBone Black, and will have its own USB camera and Ethernet connection to the cRIO for communication. One BeagleBone is dedicated to determining which goal is hot in autonomous mode. The other is dedicated to the more interesting task of autonomously catching the ball.3Locating the ballWith Dual Histogram Backprojection

abcdOriginal image, straight from the camera. For the purpose of finding the ball, it is converted to the HSV colorspace.Histogram backprojection, based on a histogram of the ball. Bright where colors closely match those of the ball from calibration images.Histogram backprojection, based on a histogram of the background. Bright where colors closely match those in the background in calibration images.Binary image that is a combination of figures b and c. Highlighted in green is the contour that is determined to most likely be the ball.

The first step in trying to autonomously catch a ball is locating it in the camera image. This is a crucial step, and I decided that I wanted a more robust algorithm than the simple threshold that we had used in past years. To make it easier to calibrate, I decided to use histogram backprojection. Then calibration consists of taking one or more images of the ball and computing a histogram. To find the likelihood that a pixel in another, later, image belongs to the ball, all you have to do is look up how many times that color pixel occurs in the histogram. Such a backprojection was used to create figure b. As you can see, this backprojection often doesnt fully recognize the ball, so to improve reliability, I also take a histogram, and perform backprojection on calibration images containing only the background. This backprojection formed figure c. To combine the two backprojections, I use the equation below; that is if a pixel in the image is more likely to be part of the ball than the background, then it is considered to be part of the ball. The gains k1 and k2 can tune the results for better reliability. Finally, because there are still a few extraneous pixels, I look for large, contiguous sets of pixels, the largest of which (if it large enough) is considered to be the ball. Figure d shows the combination of backprojections, and highlights in green the set of pixels determined to be the ball. The centroid of this set of pixels is used to track the ball.

To make implementing this algorithm much easier, I have built it on top of OpenCV, which provides functions for calculation histograms and backprojections, finding contours, etc.4Catching the Ballusing chapmans ObservationBAFig 1Adapted from Chapmans 1968 paper, Catching a Baseball

Fig 2

Once the ball has been located in the camera image, the robot must then determine how to drive in order to catch the ball. This is not an easy task, since the ball is moving in three dimensions, but the camera image is two dimensional. In 1968, however, Seville Chapman published a paper called Catching a Baseball, where he made a very useful observation. Chapman pointed out that if a ball is launched from point A to point B, then to an observer at point B, the tangent of the angle of elevation to the ball increases at a constant rate. This is illustrated in figure 1. Chapman also shows that if you start somewhere other than where the ball will land, but move at the right speed to arrive there at the same time as the ball, this observation still holds. This lends itself nicely to using the second time-derivative of that tangent as an error signal for a control loop. If the second derivative is positive, then the robot should drive away from the ball to keep up; if the second derivative is negative, then the robot should drive towards the ball to get under it; and if the second derivative is zero, then the robot has the correct velocity to catch the ball.

This is a very convenient control law, since the distance in pixels from the center of an image is proportional to the tangent of the angle from the camera to the ball. Our camera will not be pointed horizontally, however, so the angle from the camera to the ball is not the elevation angle. Instead, the angle of elevation is equal to the angle at which the camera is pointed (labled phi in figure 2), plus the angle from the camera to the ball (labled theta prime in figure 2). Then the tangent of the elevation angle theta, is equal to the tangent of phi plus theta prime, and using a trig identity, this can be reduced to an expression involving the tangent of phi, which can be computed, and the tangent of theta prime, which is equal to the distance from the center of the image to the ball over the focal length of the camera.5intro_slidenull9534.699eng - iTunPGAP0eng - iTunNORM 00000109 00000000 00000801 00000000 000006F0 00000000 00003D3E 00000000 00002178 00000000eng - iTunSMPB 00000000 00000210 00000870 0000000000066000 00000000 00016C49 00000000 00000000 00000000 00000000 00000000 00000000teamwork_slidenull51199.324eng - iTunPGAP0eng - iTunNORM 00000030 00000000 00000301 00000000 0000234E 00000000 00002757 00000000 00000463 00000000eng - iTunSMPB 00000000 00000210 000009F0 0000000000226800 00000000 0007C7D6 00000000 00000000 00000000 00000000 00000000 000000000beagle_slide0tracking_slide0chapman_slide