OpenCV - A short introduction to OpenCV librariesmoltisanti/cv1213/01a_OpenCV.pdf · A short...

OpenCVA short introduction to OpenCV libraries

Marco Moltisanti

Image Processing LabDipartimento di Matematica e Informatica

Universita degli Studi di Catania

[email protected]

March 13, 2013

Intro Resources

OpenCV (Open Source Computer Vision) is a library of programmingfunctions containing all the standard algorithms for Computer Vision;

Currently, functions are implemented in C/C++, Python and Java;

OpenCV libraries can run on Windows, Linux, Android and Macsystems;

Latest version is 2.4.4.

M. Moltisanti − OpenCV 2/27

Intro Resources

Where?

Download;

Documentation;

Tutorials.

Intro Resources

Where?

Download;

Documentation;

Tutorials.

Intro Resources

Where?

Download;

Documentation;

Tutorials.

Intro Resources

Other Resources

Book: Learning OpenCV: Computer Vision with the OpenCV Library ;

C/C++ Cheatsheet (PDF);

Library structure

Main Packages

core Basic functionalities and data structures;

imgproc Image processing functions (blurring, histograms,registration, tracking, detection);

highgui High-level Graphical User Interface;

calib3d Camera calibration and 3D Reconstruction;

features2d Features detection and description;

objdetect Object detection;

ml Machine Learning and Pattern Recognition tools (e.g.k-means, SVM, knn)

Library structure

Main Packages

Library structure

Main Packages

Library structure

Main Packages

Library structure

Main Packages

Library structure

Main Packages

Library structure

Main Packages

Library structure

Main Packages

Data structures and functions belong to the namespace cv; therefore, toaccess this functionality from your code, use the cv:: specifier or usingnamespace cv; directive

Library structure Include and libs

Directory structure

Let CVHOME be the folder where you have installed OpenCV.

CVHOME

include

opencv.....................old versions’ C/C++ header filesopencv2...................new versions’ C/C++ header files

opencv {package name } {version }d.lib........Debug libsopencv {package name } {version }.lib.......Release libs

Figure : OpenCV directory tree

Library structure Include and libs

Example

If you have installed OpenCV 2.4.2, the core functions and datastructures are declared in include/opencv2/core/core.hpp and can befound in the library opencv core 242d.lib (Debug) andopencv core 242.lib (Release)

Library structure Compiling and running

Microsoft Visual Studio 2008/2010 - Standard

The Good old method

(1) Create a new, blank, C++ Project;

(2) Open Project Properties;(3) Click on VC++ Directories;

(a) Add CVHOME/include directory to Additional Include Directories;

(4) Click on Linker ;(a) Click on General, then add CVHOME/lib directory to Additional

Library Directories;(b) Click on Input, then add the lib filenames you need to Additional

dependencies.

Microsoft Visual Studio 2008/2010 - Local Property Sheet

Local method

(1) Create a new, blank, C++ Project;

(2) Open the Property Manager ;

(3) Right click on Debug, then click on Add New Property Sheet;

(4) Apply the Good old method and save the property sheet;

(5) Do steps 2 and 3 for Release;

(6) Next time you want to create a new OpenCV-based project, justclick on Add Existing Property Sheet.

Microsoft Visual Studio 2008/2010 - Global Property Sheet

Global method

Visual Studio 2008(1) Go to Tools → Options → Projects and Solutions;(2) Apply the Good old method.

Visual Studio 2010(1) Apply the local method to the Global Property Sheet (it should be

named Microsoft.Cpp.Win32.user).

Remember that OpenCV libraries for Debug and Release differsonly in the final d!Be careful ;-)

Library structure Basic classes

Geometric primitives - Points and Rectangles

Point Template class that represents a 2-column vector containingthe coordinates of a point in a plane;

Implemented classes:

Point, Point2i: integer coordinates;Point2d: double coordinates;Point2f: float coordinates.

Point3 Template class that represents a 3-column vector containingthe coordinates of a point in the space;

Point3, Point3i: integer coordinates;Point3d: double coordinates;Point3f: float coordinates.

Rect Template class that represents a rectangle, defined by theupper-left corner coordinates, width and height.

Geometric primitives - Example

1 . . .2 P o i n t p1 ( 1 0 , 10) ;3 P o i n t 2 i p2 ( 1 1 , 11) ;4 Point2d p3 ( 1 . 0 , 1 . 0 ) ;5 P o i n t 2 f p4 ( 2 . 0 f , 2 . 0 f ) ;6

7 P o i n t 3 i p5 ( 1 0 , 10 , 10) ;8 Point3d p6 ( 1 . 0 , 1 . 0 , 1 . 0 ) ;9 P o i n t 3 f p7 ( 1 . 0 f , 1 . 0 f , 1 . 0 f ) ;

11 Rect r1 ( 1 , 1 , 100 , 100) ;12 Rect r2 ( p1 , p2 ) ;13 Rect <double> r3 ( p3 , p3 ∗1 0 . 0 ) ;14 . . .

Listing 1: Ex01.cpp

Class Mat

Mat is the fundamental class for doing everything in OpenCV. Mat canrepresent:

a matrix;

a filter;

an image;

a set of vectors (e.g. descriptors);

Class Mat

a matrix;

a filter;

an image;

Class Mat

a matrix;

a filter;

an image;

Class Mat

a matrix;

a filter;

an image;

Class Mat

a matrix;

a filter;

an image;

Class Mat

a matrix;

a filter;

an image;

A Mat object should always be released before exiting the program usingthe Mat::release() function!

Mat as an image

1 v o i d load Image ( ) {2

3 Mat img = imread ( ” l e n a . bmp” ) ;4 imshow ( ”Window” , img ) ;5 waitKey ( 0 ) ;6 img . r e l e a s e ( ) ;7 }

Listing 2: Ex02-Img.cpp

Mat as a matrix

1 . . .2#i n c l u d e <opencv2 / imgproc / imgproc . hpp>3 . . .4 v o i d r o t a t i o n M a t r i x ( ) {5 Mat r o t ;6 r o t = g e t R o t a t i o n M a t r i x 2 D ( P o i n t 2 f ( 0 . 0 f , 0 . 0 f ) , 3 0 . 0 ,

1 . 0 ) ;7

8 f o r ( i n t i =0; i < r o t . rows ; i ++) {9 f o r ( i n t j =0; j < r o t . c o l s ; j ++)

10 cout << r o t . at<double >( i , j ) << ”\ t ” ;11 cout << e n d l ;12 }13

14 }Listing 3: Ex02-Mat.cpp

Mat as a filter

1 . . .2#i n c l u d e <opencv2 / imgproc / imgproc . hpp>3 . . .4 v o i d c r e a t e F i l t e r ( ) {5

6 Mat f l t ;7 f l t = g e t G a u s s i a n K e r n e l ( 5 , 2 . 0 ) ;8

9 f o r ( i n t i =0; i < f l t . rows ; i ++) {10 f o r ( i n t j =0; j < f l t . c o l s ; j ++)11 cout << f l t . at<double >( i , j ) << ”\ t ” ;12 cout << e n d l ;13 }14 }

Listing 4: Ex02-Flt.cpp

Mat as a set

1 v o i d c r e a t e O b s e r v a t i o n s S e t ( ) {2 Mat data ( 1 0 , 5 , CV 32S ) ;3 f o r ( i n t i =0; i <10; i ++) {4 f o r ( i n t j =0; j< 5 ; j ++)5 data . at( i , j ) = rand ( ) ∗2 5 6 ;6

9 f o r ( i n t i =0; i < data . rows ; i ++) {10 f o r ( i n t j =0; j < data . c o l s ; j ++)11 cout << data . at( i , j ) << ”\ t ” ;12 cout << e n d l ;13 }14 }

Listing 5: Ex02-Set.cpp

Notes on previous slides

As you probably noticed, we used two functions:

(1) getRotationMatrix2D;

(2) getGaussianKernel.

Both belong to imgproc subset, so we included the correspondent headerfile (see source code for details).

Graphic User Interface

Loading, Showing and Writing an image

All graphical user interface (GUI) utilities are defined in highgui

library;

Include directive: opencv2\highgui\highgui.hpp;

imread(filename): reads the image from the filename passed as aparameter. Returns a Mat;

imshow(windowname, image): show the image in a new windowtitled windowname;

imwrite(filename, image, params): write the image in a new filenamed filename using the format specific params. The format isderived from the extension given in the filename.

library;

1 . . .2 // Read t he image3 cv : : Mat l e n a = cv : : imread ( ” l e n a . bmp” ) ;4

5 //Show t he image6 cv : : imshow ( ” Lena ” , l e n a ) ;7 cv : : waitKey ( 0 ) ;8

9 // Set params10 s t d : : v e c t o r jpg param ;11 jpg param . push back ( CV IMWRITE JPEG QUALITY ) ;12 jpg param . push back ( 5 0 ) ;13

14 // Wri te t he image15 cv : : i m w r i t e ( ” l e n a . j p g ” , l en a , jpg param ) ;16 . . .

Listing 6: Ex03.cpp

Image Processing

Apply a built-in filter

Include directive: opencv2\imgproc\imgproc.hpp;

Load the image;

Create the output image;

Create and apply the filter;

Show the image;

Save the image.

Image Processing

Load the image;

Show the image;

Save the image.

Image Processing

Load the image;

Show the image;

Save the image.

Image Processing

Load the image;

Show the image;

Save the image.

Image Processing

Load the image;

Show the image;

Save the image.

Image Processing

Load the image;

Show the image;

Save the image.

Image Processing

1 . . .2 // Read t he image3 Mat l e n a = imread ( ” l e n a . bmp” ) ;4

5 //Show t he o r i g i n a l image6 imshow ( ” Lena o r i g i n a l ” , l e n a ) ;7

8 // C r e a t e th e output image9 Mat l e n a c a n n y = l e n a . c l o n e ( ) ;

11 . . .12

13 // C r e a t e and a p p l y th e f i l t e r14 Canny ( l e na , l e n a c a n n y , m[ 0 ] ∗ 0 . 6 6 , m[ 0 ] ∗ 1 . 3 3 ) ;15 . . .

Listing 7: Ex04.cpp

Image Processing Video Handling

Video from device

Video handling is included in highgui package;

Declare a VideoCapture(id) object; id is the camera identificationnumber;

Enter a loop;

In each iteration:

Grab a frame using the >> operator and store it in a Mat object;Check if the frame is empty;;Show the frame;

Video from device

Enter a loop;

In each iteration:

Video from device

Enter a loop;

In each iteration:

Video from device

Enter a loop;

In each iteration:

Video from device

Enter a loop;

In each iteration:

Video from device

Enter a loop;

In each iteration:

Video from device

Enter a loop;

In each iteration:

Video from device

1 VideoCapture vc ( 0 ) ;2 c h a r r e s = ’ 0 ’ ;3 i n t f p s = vc . g e t ( CV CAP PROP FPS ) ;4 Mat frame ;5 w h i l e ( 1 ) {6 vc >> f rame ;7 i f ( f rame . empty ( ) )8 b r e a k ;9 imshow ( ” Video ” , f rame ) ;

10 r e s = ( c h a r ) ( waitKey ( 0 ) ) ;11

12 i f ( r e s == ’ q ’ )13 b r e a k ;14 }15 vc . r e l e a s e ( ) ;

Listing 8: Ex05-In.cpp

Video from file

The difference between using a video from file and a video from a device isonly in the constructor. Instead of VideoCapture(id) function, we willuse the overloaded version VideoCapure(filename), where filename isthe path to a video file.

Example

VideoCapture vc(”video.mp4”)

Save a video

Declare a VideoWriter objectand set the properties for the newvideo using the constructor;

Put the frames in the VideoWriter object using the << operator;

Release the VideoWriter object.

Save a video

1 S i z e s ( vc . g e t (CV CAP PROP FRAME WIDTH) , vc . g e t (CV CAP PROP FRAME HEIGHT) ) ;

2 V i d e o W r i t e r vw ( ” v ideoOut . a v i ” , vc . g e t (CV CAP PROP FOURCC), vc . g e t ( CV CAP PROP FPS ) , s , t r u e ) ;

4 w h i l e ( 1 ) {5

6 . . .7 f l i p ( frame , newFrame , 1) ;8 vw << newFrame ;9 imshow ( ” I t ’ s someth ing ! ” , newFrame ) ;

10 r e s = ( c h a r ) waitKey (1000/ f p s ) ;11 . . .12 }

Listing 9: Ex05-Out.cpp

Machine Learning

Available methods

Statistical Models;

Normal Bayes Classifier;

Decision Trees;

Boosting;

Gradient Boosted Trees;

Random Trees;

EM algorithm;

Neural Networks.

Machine Learning

Available methods

Statistical Models;

Decision Trees;

Boosting;

Random Trees;

EM algorithm;

Neural Networks.

Machine Learning

Available methods

Statistical Models;

Decision Trees;

Boosting;

Random Trees;

EM algorithm;

Neural Networks.

Machine Learning

Available methods

Statistical Models;

Decision Trees;

Boosting;

Random Trees;

EM algorithm;

Neural Networks.

Machine Learning

Available methods

Statistical Models;

Decision Trees;

Boosting;

Random Trees;

EM algorithm;

Neural Networks.

Machine Learning

Available methods

Statistical Models;

Decision Trees;

Boosting;

Random Trees;

EM algorithm;

Neural Networks.

Machine Learning

Available methods

Statistical Models;

Decision Trees;

Boosting;

Random Trees;

EM algorithm;

Neural Networks.

Machine Learning

Available methods

Statistical Models;

Decision Trees;

Boosting;

Random Trees;

EM algorithm;

Neural Networks.

Machine Learning

Available methods

Statistical Models;

Decision Trees;

Boosting;

Random Trees;

EM algorithm;

Neural Networks.

Machine Learning

Available methods

Statistical Models;

Decision Trees;

Boosting;

Random Trees;

EM algorithm;

Neural Networks.

Machine Learning

Available methods

Include Directive

Include path: <opencv2\ml\ml.hpp>Documentation is fundamental using ml functions!

In conclusion...

Tips & Tricks

Tip #1

Documentation is your biggest friend while developing!

Tip #2

Community is your second biggest friend!Google to find out how your colleagues all around the world solved theirproblems!

Tip #3

Don’t try to copy!

In conclusion...

If you have doubts don’t hesitate to contact me at:

[email protected];

IPLab (room 146 - ground floor);

Room 31 - first floor in the classrooms area.

Slides available at

Course page

Personal page

Code available here - Personal page

In conclusion...

Question time!

OpenCV - A short introduction to OpenCV librariesmoltisanti/cv1213/01a_OpenCV.pdf · A short...

Documents

Transcript of OpenCV - A short introduction to OpenCV librariesmoltisanti/cv1213/01a_OpenCV.pdf · A short...