VideoForge An image processing system. Contents Current state of the art IntroductionTiers Base Base...
-
Upload
adelia-day -
Category
Documents
-
view
212 -
download
0
Transcript of VideoForge An image processing system. Contents Current state of the art IntroductionTiers Base Base...
VideoForgeVideoForge
An image processing systemAn image processing system
ContentsContents
Current state of the artCurrent state of the artIntroductionIntroductionTiersTiers
BaseBase CoreCore ModulesModules
User interfaceUser interface Interactive modeInteractive mode Scripting Scripting
universal interfaceuniversal interfaceprimary implementation COM scripting objectprimary implementation COM scripting object
Implementation templatesImplementation templates ControllersControllers AlgorithmsAlgorithms
Current state of the artCurrent state of the art
LibrariesLibraries Loki, BoostLoki, Boost Direct Show™, ACML, IPL(IPP), OpenCV, Microsoft Direct Show™, ACML, IPL(IPP), OpenCV, Microsoft
Vision SDK, ImageMagick, …Vision SDK, ImageMagick, …
ApplicationsApplications GraphEdit (part of the DirectShow)GraphEdit (part of the DirectShow) PhotoshopPhotoshop ImageJImageJ ImageForgeImageForge Adobe Premiere, …Adobe Premiere, …
IntroductionIntroduction
System for video and live-stream image System for video and live-stream image processingprocessing
Platform independentPlatform independent ANSI C++ANSI C++
Easy-to-do computer visionEasy-to-do computer vision
Three tier applicationThree tier application Base (Serialization, Communication, Logging)Base (Serialization, Communication, Logging) Core (Module management, Memory management)Core (Module management, Memory management) Modules (Devices, Algorithms,)Modules (Devices, Algorithms,)
BaseBase
SerializationSerialization Binary vs. textual storageBinary vs. textual storage Automatic vs. manual storageAutomatic vs. manual storage Type lists, storing of arrays, classes …Type lists, storing of arrays, classes …
Variable reflectionVariable reflection offers the possibility of calling the variable “by offers the possibility of calling the variable “by
name”name” m_plugin.FindVar<int>(“varName”);m_plugin.FindVar<int>(“varName”); m_plugin.GetVarNames(…);m_plugin.GetVarNames(…);
User interfaceUser interface
CommunicationCommunication Scripting possibilityScripting possibility User interface (Interactive mode)User interface (Interactive mode)
Types of communicationTypes of communication Actions – non-parametric methodsActions – non-parametric methods Properties – like variables, accessor methodsProperties – like variables, accessor methods Methods – methods with parametersMethods – methods with parameters Events – (e.g. onClose, …)Events – (e.g. onClose, …)
CoreCore
Module and plug-in managementModule and plug-in management EnumerationEnumeration RegistrationRegistration
Memory managementMemory management Memory (de-)allocationMemory (de-)allocation Correct memory handling for erroneous Correct memory handling for erroneous
modulesmodules
Actions (from plug-ins and core itself)Actions (from plug-ins and core itself) User interface handlingUser interface handling
Modules and plug-insModules and plug-ins
Module – fileModule – file Contains plug-in implementationsContains plug-in implementations compiled-in or represented by DLLcompiled-in or represented by DLL
Plug-insPlug-ins SourcesSources
Feeds data into chainFeeds data into chain AlgorithmsAlgorithms
Transforms data, somehowTransforms data, somehow Special plug-insSpecial plug-ins
usually compiled : persistence, etc.usually compiled : persistence, etc.
Image & FrameImage & Frame
similar to IPL IMAGE structuresimilar to IPL IMAGE structure
contains:contains: resolutionresolution ROI (rectangle of interest)ROI (rectangle of interest) MediaTypeMediaType
color mode (rgb, YUV, etc.) color mode (rgb, YUV, etc.)
bpp (=color depth)bpp (=color depth)
FrameFrame time information (start, stop time, fps)time information (start, stop time, fps) previous & next frame reference (=pointer)previous & next frame reference (=pointer)
Building the graphBuilding the graph
Pipeline buildingPipeline building Interactive mode vs. non-interactive modeInteractive mode vs. non-interactive mode
Device (Controller = Device enumerator)Device (Controller = Device enumerator)
Algorithms (Algorithm chain = Composite algs)Algorithms (Algorithm chain = Composite algs)
System windowsSystem windows
WindowsWindows show graphical informationshow graphical information part of a particular pluginpart of a particular plugin
Bitmap windowsBitmap windows shows contents of an image buffershows contents of an image buffer
Vector windowsVector windows usually used to show some information in the imageusually used to show some information in the image applied as a second layer onto the bitmap windowapplied as a second layer onto the bitmap window
Window eventsWindow events OnClose, …OnClose, …
Templates (1/3)Templates (1/3)
Application templateApplication template UI applicationUI application Console applicationConsole application
Module templateModule template default implementationdefault implementation rare modificationrare modification implements a list of plug-insimplements a list of plug-ins
Templates (2/3)Templates (2/3)
Controller templateController template Implement source enumerationImplement source enumeration Implement data feedImplement data feed Possibly implement UIPossibly implement UI
Algorithm templateAlgorithm template Implement data transformationImplement data transformation Possibly implement UIPossibly implement UI User must implement 1 methodUser must implement 1 method
ConclusionConclusion
We proposed system for implementation of We proposed system for implementation of image processing and computer vision tasksimage processing and computer vision tasks as platform-independent as possible (ANSI C++)as platform-independent as possible (ANSI C++) consists of (lightweight) core and pluginsconsists of (lightweight) core and plugins offers a user interactive, scriptable user interfaceoffers a user interactive, scriptable user interface offers various templates for fast implementation offers various templates for fast implementation
The system is a part of various grantsThe system is a part of various grants Grant UKGrant UK APVV GrantAPVV Grant
Future workFuture work
Finalize implementation of the core with the Finalize implementation of the core with the basic modules (12/05)basic modules (12/05) basic image processing operationsbasic image processing operations
Implementation of basic modules of 0(1/06)Implementation of basic modules of 0(1/06) working set optimizationworking set optimization
Implementation of special modules (06/06)Implementation of special modules (06/06) Feature trackingFeature tracking Cooperation with avatarsCooperation with avatars Statistics – classifier trainingStatistics – classifier training