bw = imread('box.jpg');bw=im2bw(bw);imshow(bw)L = bwlabel(bw);s = regionprops(L, 'Area', 'BoundingBox');s(1);area_values = [s.Area];idx = find((100 <= area_values) & (area_values <= 1000)); % list of all the objects

%whose area is between 100 and 1000

bw2 = ismember(L, idx); %construct a binary image containing all the objects whose

%area is between 100 and 1000 by passing L and idx to ismember.

imshow(bw2)I treid this after the above code. But its not a good way of doing this: imshow(bw2) rectangle('Position', [ s(2).Centroid+35, 70, 100], 'EdgeColor','r') 

--------------------------------------------------------------------------

Pseduo -

Pick largest y, largest x, smallest x, smallest y with in the blob. That is, points on the blob. These are your coordinates that you can use to build the bounding box.

assuming top left of image as (0,0)

(smallestX,smallestY)-----------------(largestX,smallestY) | | | | | | | |(smallestX,largestY)------------------(largestX,largestY) And for finding minimum/maximum values and indices.

[r,c]=find(img==min(min(img)))[r,c]=find(img==max(max(img)))r,c represent row and column in the img matrix.

I have marked the points on your image that you can use to create the bounding box. Zoomed Image to get a better view. 

I think you can try to use bwboundriesboundaries = bwboundaries(blob); numberOfBoundaries = size(boundaries); for k = 1 : numberOfBoundaries thisBoundary = boundaries{k}; plot(thisBoundary(:,2), thisBoundary(:,1), 'g', 'LineWidth', 2); end

Find the corners of a polygon represented by a region maskBW = poly2mask(x, y, m, n) computes a binary region of interest (ROI) mask, BW, from an ROI polygon, represented by the vectors x and y. The size of BW is m-by-n.poly2mask sets pixels in BW that are inside the polygon (X,Y) to 1 and sets pixels outside the polygon to 0.

Problem: Given such a binary mask BW of a convex quadrilateral, what would be the most efficient way to determine the four corners?E.g.,

Best Solution so far: Use edge to find the bounding lines, the Hough transform to find the 4 lines in the edge image and then find the intersection points of those 4 lines or use a corner detector on the edge image. Seems complicated, and I can't help feeling there's a simpler solution out there.Btw, convhull doesn't always return 4 points (maybe someone can suggest qhull options to prevent that) : it returns a few points along the edges as well.EDIT: Amro's answer seems quite elegant and efficient. But there could be multiple "corners" at each real corner since the peaks aren't unique. I could cluster them based on θ and average the "corners" around a real corner but the main problem is the use of order(1:10).Is 10 enough to account for all the corners or will this exclude a "corner" at a real corner?This is somewhat similar to what @AndyL suggested. However I'm using the boundary signature in polar coordinates instead of the tangent.Note that I start by extracting the edges, getting the boundary, then converting it to signature. Finally we find the points on the boundary that are furthest from the centroid, those points constitute the corners found. (Alternatively we can also detect peaks in the signature for corners).

The following is a complete implementation:

I = imread('oxyjj.png');if ndims(I)==3 I = rgb2gray(I);endsubplot(221), imshow(I), title('org')

%%# Process Image%# edge detectionBW = edge(I, 'sobel');subplot(222), imshow(BW), title('edge')

%# dilation-erosionse = strel('disk', 2);BW = imdilate(BW,se);BW = imerode(BW,se);subplot(223), imshow(BW), title('dilation-erosion')

%# fill holesBW = imfill(BW, 'holes');subplot(224), imshow(BW), title('fill')

%# get boundaryB = bwboundaries(BW, 8, 'noholes');B = B{1};

%%# boudary signature%# convert boundary from cartesian to ploar coordinatesobjB = bsxfun(@minus, B, mean(B));[theta, rho] = cart2pol(objB(:,2), objB(:,1));

%# find corners%#corners = find( diff(diff(rho)>0) < 0 ); %# find peaks[~,order] = sort(rho, 'descend');corners = order(1:10);

%# plot boundary signature + cornersfigure, plot(theta, rho, '.'), hold onplot(theta(corners), rho(corners), 'ro'), hold offxlim([-pi pi]), title('Boundary Signature'), xlabel('\theta'), ylabel('\rho')

%# plot image + cornersfigure, imshow(BW), hold onplot(B(corners,2), B(corners,1), 's', 'MarkerSize',10, 'MarkerFaceColor','r')hold off, title('Corners')-------------------------

 

EDIT: In response to Jacob's comment, I should explain that I first tried to find the peaks in the signature using first/second derivatives, but ended up taking the furthest N-points. 10 was just an ad-hoc value, and would be difficult to generalize (I tried taking 4 same as number of corners, but it didn't cover all of them). I think the idea of clustering them to remove duplicates is worth looking into.As far as I see it, the problem with the 1st approach was that if you plot rho without taking θ into account, you will get a different shape (not the same peaks), since the speed by which we trace the boundary is different and depends on the curvature. If we could figure out how to normalize that effect, we can get more accurate results using derivatives.

If you have the Image Processing Toolbox, there is a function called CORNERMETRIC which can implement a Harris corner detector or Shi and Tomasi's minimum eigenvalue method. This function has been present since version 6.2 of the Image Processing Toolbox (MATLAB version R2008b).Using this function, I came up with a slightly different approach from the other answers. The solution below is based on the idea that a circular area centered at each "true" corner point will overlap the polygon by a smaller amount than a circular area centered over an erroneous corner point that is actually on the edge. This solution can also handle cases where multiple points are detected at the same corner...

The first step is to load the data:

rawImage = imread('oxyjj.png');rawImage = rgb2gray(rawImage(7:473,9:688,:)); %# Remove the gray bordersubplot(2,2,1);imshow(rawImage);title('Raw image');Next, compute the corner metric using CORNERMETRIC. Note that I am masking the corner metric by the original polygon, so that we are looking for corner points that are inside the polygon (i.e. trying to find the corner pixels of the polygon). IMREGIONALMAX is then used to find the local maxima. Since you can have clusters of greater than 1 pixel with the same corner metric, I then add noise to the maxima and recompute so that I only get 1 pixel in each maximal region. Each maximal region is then labeled using BWLABEL:cornerImage = cornermetric(rawImage).*(rawImage > 0);maxImage = imregionalmax(cornerImage);noise = rand(nnz(maxImage),1);cornerImage(maxImage) = cornerImage(maxImage)+noise;maxImage = imregionalmax(cornerImage);labeledImage = bwlabel(maxImage);The labeled regions are then dilated (using IMDILATE) with a disk-shaped structuring element (created usingSTREL):diskSize = 5;dilatedImage = imdilate(labeledImage,strel('disk',diskSize));subplot(2,2,2);imshow(dilatedImage);title('Dilated corner points');Now that the labeled corner regions have been dilated, they will partially overlap the original polygon. Regions on an edge of the polygon will have about 50% overlap, while regions that are on a corner will have about 25% overlap. The function REGIONPROPS can be used to find the areas of overlap for each labeled region, and the 4 regions that have the least amount of overlap can thus be considered as the true corners:maskImage = dilatedImage.*(rawImage > 0); %# Overlap with the polygonstats = regionprops(maskImage,'Area'); %# Compute the areas[sortedValues,index] = sort([stats.Area]); %# Sort in ascending ordercornerLabels = index(1:4); %# The 4 smallest region labelsmaskImage = ismember(maskImage,cornerLabels); %# Mask of the 4 smallest regionssubplot(2,2,3);imshow(maskImage);title('Regions of minimal overlap');And we can now get the pixel coordinates of the corners using FIND and ISMEMBER:[r,c] = find(ismember(labeledImage,cornerLabels));subplot(2,2,4);imshow(rawImage);hold on;plot(c,r,'r+','MarkerSize',16,'LineWidth',2);

title('Corner points');

And here's a test with a diamond shaped region:

I like to solve this problem by working with a boundary, because it reduces this from a 2D problem to a 1D problem.

Use bwtraceboundary() from the image processing toolkit to extract a list of points on the boundary. Then convert the boundary into a series of tangent vectors (there are a number of ways to do this, one way would be to subrtact the ith point along the boundary from the i+deltath point.) Once you have a list of vectors, take the dot product of adjacent vectors. The four points with the smallest dot products are your corners!If you want your algorithm to work on polygons with an abritrary number of vertices, then simply search for dot products that are a certain number of standard deviations below the median dot product.

I decided to use a Harris corner detector (here's a more formal description) to obtain the corners. This can be implemented as follows:%% ConstantsWindow = 3;Sigma = 2;K = 0.05;nCorners = 4;

%% Derivative masksdx = [-1 0 1; -1 0 1; -1 0 1];dy = dx'; %SO code color fix '

%% Find the image gradient% Mask is the binary image of the quadrilateral

Ix = conv2(double(Mask),dx,'same'); Iy = conv2(double(Mask),dy,'same');

%% Use a gaussian windowing function and compute the restGaussian = fspecial('gaussian',Window,Sigma);Ix2 = conv2(Ix.^2, Gaussian, 'same'); Iy2 = conv2(Iy.^2, Gaussian, 'same');Ixy = conv2(Ix.*Iy, Gaussian, 'same');

%% Find the cornersCornerStrength = (Ix2.*Iy2 - Ixy.^2) - K*(Ix2 + Iy2).^2;[val ind] = sort(CornerStrength(:),'descend'); [Ci Cj] = ind2sub(size(CornerStrength),ind(1:nCorners));

%% Displayimshow(Mask,[]);hold on;plot(Cj,Ci,'r*');Here, the problem with multiple corners thanks to Gaussian windowing function which smooths the intensity change. Below, is a zoomed version of a corner with the hot colormap.

Table 1

Nope. Listen I was testing your approach, and it seems it's not perfect either (which is always the case in Computer Vision!). Try the following mask with your code above: x = [16 282 276 30 16]; y = [14 29 200 225 14]; BW = poly2mask(x,y, 246,300); you will get some duplicate corners (even when tried tuning the other params), and will have to increase nCorners too.. –  Amro Nov 12 '09 at 21:48

You're completely right - looks like clustering is required for some polygons. –  Jacob Nov 12 '09 at 22:58

I don't want to push my idea too strongly, but I think that the signature method could be perfected for most quadrilateral if we locate the exact 4 theta peaks, which are after all clearly visible in the plot. Picking the highest N-points may not have been the best way, but maybe someone can improve that part

----------------------------------------------------

Here's an example using Ruby and HornetsEye. Basically the program creates a histogram of the quantised Sobel gradient orientation to find dominant orientations. If four dominant orientations are found, lines are fitted and the intersections between neighbouring lines are assumed to be the corners of the projected rectangle.

#!/usr/bin/env rubyrequire 'hornetseye'include HornetseyeQ = 36img = MultiArray.load_ubyte 'http://imgur.com/oxyjj.png'dx, dy = 8, 6box = [ dx ... 688, dy ... 473 ]crop = img[ *box ]crop.shows0, s1 = crop.sobel( 0 ), crop.sobel( 1 )mag = Math.sqrt s0 ** 2 + s1 ** 2mag.normalise.showarg = Math.atan2 s1, s0msk = mag >= 500arg_q = ( ( arg.mask( msk ) / Math::PI + 1 ) * Q / 2 ).to_int % Qhist = arg_q.hist_weighted Q, mag.mask( msk )segments = ( hist >= hist.max / 4 ).componentslines = arg_q.map segmentslines.unmask( msk ).normalise.showif segments.max == 4 pos = MultiArray.scomplex *crop.shape

pos.real = MultiArray.int( *crop.shape ).indgen! % crop.shape[0] pos.imag = MultiArray.int( *crop.shape ).indgen! / crop.shape[0] weights = lines.hist( 5 ).major 1.0 centre = lines.hist_weighted( 5, pos.mask( msk ) ) / weights vector = pos.mask( msk ) - lines.map( centre ) orientation = lines.hist_weighted( 5, vector ** 2 ) ** 0.5 corner = Sequence[ *( 0 ... 4 ).collect do |i| i1, i2 = i + 1, ( i + 1 ) % 4 + 1 l1, a1, l2, a2 = centre[i1], orientation[i1], centre[i2], orientation[i2] ( l1 * a1.conj * a2 - l2 * a1 * a2.conj - l1.conj * a1 * a2 + l2.conj * a1 * a2 ) / ( a1.conj * a2 - a1 * a2.conj ) end ] result = MultiArray.ubytergb( *img.shape ).fill! 128 result[ *box ] = crop corner.to_a.each do |c| result[ c.real.to_i + dx - 1 .. c.real.to_i + dx + 1, c.imag.to_i + dy - 1 .. c.imag.to_i + dy + 1 ] = RGB 255, 0, 0 end result.showend

-----------------------------------------------------------------------------

x=[ 39 63 71 7 53 39 64 23 29 65 ];

y=[ 20 11 22 78 61 71 9 20 78 94 ];

AA=[x ; y]

% Find point in AA closest to p1.

p1 = [39,20]

squaredDistance = sum((AA-repmat(p1', [1, size(AA, 2)])).^2, 1)

[maxSqDist1, indexOfMax1] = max(squaredDistance)

% Find point in AA closest to p2.

p2 =[63,11]

squaredDistance = sum((AA-repmat(p2', [1, size(AA, 2)])).^2, 1)

[maxSqDist2, indexOfMax2] = max(squaredDistance)

-----------------------------------------------------------------------------

By following this post Using ismember with the output of regionprops I am able to selectively isolate the connected component I want. For example using my code below:img = rgb2gray(imread('W1\Writer1_01_02.jpg'));bw_normal2 = im2bw(img, graythresh(img));bw22 = imcomplement(bw_normal2);bw3 = bwmorph(bw22, 'dilate');[label2,n2] = bwlabel(bw3);stats2 = regionprops(label2, {'Area', 'BoundingBox'});area2 = [stats2.Area];

idx = find((28 <= area2) & (area2 <= 40)); BW2 = ismember(label2,idx);figure, imshow(BW2)I can easily display the output that contains ONLY the connected component whose area is between 28 and 40. As so

But instead can I make a bounding box around this connected component in the original image. I mean if this is

the original image: 

Can I make a bounding box around my desired component on the original image? I know that this is the code for making a bounding box around all of my connected components

imshow(img);for j=1:n2

hold on rectangle('Position',[stats(j).BoundingBox(1),stats(j).BoundingBox(2),stats(j).BoundingBox(3),stats(j).BoundingBox(4)],...'EdgeColor','r','LineWidth',2 );endBut how do I only make a bounding box only around the element having area between 28 and 40? Rather than producing a completely different image as shown above.

Table 2

 might not have understood the question properly. But it seems like that you have the following piece of codefor j=1:n2 which would mean that you are traversing each of the component on the image. Find the number which represents the object of interest and draw the bounding box. –  Lokesh A. R. Feb 24 at 16:24

That would be quite difficult and time consuming. Say I only want the object having an area of 40. I would then open the stats2.Area in my workspace, find it and then apply it to my code. I just added that second code to show that I know how to draw a bounding box around all components but how to draw on the original image for the element selected by ismember? –  Faraz Khan Feb 24 at 16:54

1

Er, what's difficult about stats2(idx).BoundingBox? If you're not clear how the code works, try stepping through in the debugger to see what the values are, and how they change, line-by-line (use "step over" rather than "step into" to avoid the guts of all the functions)

Keep a if condition in your second half of the code...

imshow(img);for j=1:n2 hold on area2 = stats2(j).Area; if((28 <= area2) & (area2 <= 40)) rectangle('Position',[stats2(j).BoundingBox(1),stats2(j).BoundingBox(2),stats2(j).BoundingBox(3),stats2(j).BoundingBox(4)],...'EdgeColor','r','LineWidth',2 ); endend-------------------------------------------------------------------------------------

How to automatically identify text lines from projection plot?Asked by Faraz on 18 Jan 2014Latest activity Answered by fawzi about 14 hours agoAccepted Answer by Image AnalystI have been reading about automatic text line recognition in Matlab and although there are many advanced methods to do this every paper mentions that the simplest way of detecting text lines is via horizontal projections. So I decided to try this method for myself.

I am very new to this and have hit a brick wall, I have reached a level beyond which I do not know how to proceed. This is what I have achieved so far:

I'm trying for a system that is language independent and only interested in text lines, so I chose Arabic text:

I used the function ``radon`` to get the projections.

img = rgb2gray(imread('arabic.jpg'));

[R, xp] = radon(bw_closed, [0 90]);

figure; plot(xp,R(:,2)); title('at angle 90');

This is the plot(projection)

So clearly the 5 peaks represent the 5 lines detected but how do I go from here to segmenting the original document?

Can anyone help me beyond this point? All the papers I read make no mention of how to proceed from this step, they just say that from the projections we have our detected lines.

What I'm asking is how, from the plot data can I tell matlab what is the line of text and what is the gab between lines?

I would just find where the black is in the profile

darkPixels = R < 20; % Threshold

% label

labeledRegions = bwlabel(darkPixels);

% Find centroids

measurements = regionprops(labeledRegions, 'Centroid');

% Get them into an array

allCentroids = [measurements.Centroid];

Now you can just crop out some line of text you're interested in, into a separate image:

thisLine = yourImage(allCentroids(k):allCentroids(k+1), :);

-------------------------------------------------------------------------------------------------------------------

Image projection in Matlab with Hough transform

I am using Matlab to input a 4x4 grid of coloured squares, and output a list of colours. My program works fine for squares, but I am having trouble adapting it to rotated and projected images. I have been advised to use Hough transforms, and I am able to use this to access the lines in the image with the following code:[H, theta, rho] = hough(image,'RhoResolution',0.1,'Theta',-90:0.5:89.5); peaks = houghpeaks(H,4); lines = houghlines(dilated, theta, rho, peaks, 'MinLength', 40)

figure, imshow(dilated), hold on;

max_len = 0;

for k = 1:length(lines)

xy = [lines(k).point1; lines(k).point2];

plot(xy(:,1),xy(:,2),'LineWidth',2,'Color','green');

% Plot beginnings and ends of lines plot(xy(1,1),xy(1,2),'x','LineWidth',2,'Color','yellow'); plot(xy(2,1),xy(2,2),'x','LineWidth',2,'Color','red');

angle = atand(((xy(1,1)-xy(2,1))/(xy(1,2)-xy(2,2))));

end

So now I would like to use these lines, and straighten my image accordingly, so that image edges are parallel with these lines. However, I don't know how to go about this. I would appreciate advice on how to do this. Thanks.

Detecting intersecting lines after a hough transformUsing Hough transform in Matlab,detected some lines. Using the end points of these lines I have plotted them. I cant understand how I can find intersecting lines when I have all the variables.

Line7Point1 [50,66]Point2 [11,106]theta,rho [45,81]

Line9Point1 [19,83]Point2 [53,79]theta,rho [82,84]Since the parametric equations are as follows

rho = xCos(theta) + ySin(theta)I am unsure how to solve this. With all this information there must be a quick way of finding if the lines intersect, if so, the points as well.

Any guidance much appreciated.

function FindHoughLines(I,filename)[H,T,R] = hough(I);rotI = imrotate(I,0,'crop');imshow(H,[],'XData',T,'YData',R,... 'InitialMagnification','fit');xlabel('\theta'), ylabel('\rho');axis on, axis normal, hold on;P = houghpeaks(H,10,'threshold',ceil(0.1*max(H(:))));x = T(P(:,2)); y = R(P(:,1)); plot(x,y,'s','color','white');% Find lines and plot themlines = houghlines(I,T,R,P,'FillGap',5,'MinLength',7); figure, imshow(rotI), hold onmax_len = 0;for k = 1:length(lines) if(isField(lines,'point1') ~= 0) xy = [lines(k).point1; lines(k).point2]; plot(xy(:,1),xy(:,2),'LineWidth',2,'Color','green');

% Plot beginnings and ends of lines plot(xy(1,1),xy(1,2),'x','LineWidth',2,'Color','yellow'); plot(xy(2,1),xy(2,2),'x','LineWidth',2,'Color','red');

text(xy(1,1),xy(1,2),[ num2str(k)],'HorizontalAlignment','center','BackgroundColor',[.7 .9 .7]); % Determine the endpoints of the longest line segment len = norm(lines(k).point1 - lines(k).point2); if ( len > max_len) max_len = len; xy_long = xy; end endend

Are you wanting intersections of lines or line segments? For instance, suppose one of your line segments has endpoints (1,0) and (2,0) and the other has endpoints (0,1) and (0,2); do you want the intersection at (0,0) counted or not?Do you need to cope efficiently with the case where there are many many lines and find all their intersections, or is it OK to consider all pairs of lines and check them one by one?

The easiest case is if it's OK just to consider pairs of lines, and if line (as opposed to line segment) intersections are good enough. Then for each pair of lines you're just solving two simultaneous linear equations: aX+bY=c, dX+eY=f. This is a very standard thing to do; it amounts to inverting a 2x2 matrix.

If you need to notice when an intersection doesn't actually lie within the given line segments, here are a couple of approaches. (1) First first the intersection as above, then check that it lies within each given segment. You can do that by picking one coordinate (say, the x-coordinate) and seeing whether it lies between the x-coordinates of the two endpoints. Of course you can't use the x-coordinate for vertical lines and shouldn't use it for nearly-vertical lines; better to pick whichever coordinate has the larger coefficient. (2) Write the lines parametrically as (x,y)=(x1,y1)+t(dx1,dx1) and (x,y)=(x2,y2)+u(dx2,dy2); now instead of simultaneous equations for x,y you have simultaneous equations for t,u; solve those and check that 0 <= t,u <= 1.

If you need to cope efficiently when you have many lines, there are algorithms for this; google "line sweep" to find a standard one that's pretty easy to understand.