Visual Geometry Group Department of Engineering Science...
Transcript of Visual Geometry Group Department of Engineering Science...
Image Classification
Dr. Ernesto CotoResearch Software Engineer
Visual Geometry GroupDepartment of Engineering Science, University of Oxford
Outline
2
BBC News Search
– Image Classification Engine
–Demo
Sharing our work
–VGG's Image Classification (VIC) engine
–Demo
– Features
–Availability
–Deployment
–Data Ingestion
BBC News Search
Powerful public web engine for visual search overthe BBC News dataset ( ~17K programmes, ~10Khours of video, ~5M keyframes, ~1.5 TB)
3Available at: http://zeus.robots.ox.ac.uk/bbc_search/
BBC News Search
Objects/Scenes (Image Classification): Searches of abroad nature: e.g. animals (horses, etc.), vehicles (cars, etc.),styles (gothic architecture, etc.)
Instances (Image Matching): Searches for a specificobject or building (e.g Big Ben)
Text: Searches for instances of a text string
People: Searches of people or face types (e.g. Tony Blair)
4
BBC News Search
The Image Classification engine can be input text. The textis used to downloaded training images from a searchengine. This starts a search for similar images in the dataset
Matching images in the dataset are ranked and displayed
If the input is an image, the image is used to train themodel
5
Sharing our work: VIC
You can use the same Image Classification engine forsearching on your own dataset !
VGG's Image Classification (VIC) engine
6
Sample dataset: COCO2014
Common Object in Context (COCO) – 2014 version
Complex everyday scenes containing common objects intheir natural context.
Includes 80 object categories
More than 300K images
– We use a subset that includes objects and text (~14Kimages)
http://mscoco.org/7
VIC: Features
Query by entering a text or an image
Training images automatically downloaded fromGoogle
Automatic training, classification and result ranking
User management
Query refining (Uber classifiers) and caching
Curated queries
Data ingestion:
– Use your own dataset and/or extend it later
– Define your own metadata
8
VIC: Availability
Supported platforms: Linux and Mac. Currentlyworking on the Windows deployment
Deployment options:
–Simple deployment in your computer via Dockertechnology
– For advanced users, native deployment from thesource code is also possible
9Images take from: https://www.docker.com
VIC: Simple Deployment
1) Install Docker (needs privileged access)
2) Install the VIC application (installer files provided)
3) Instruct VIC to ingest your data
10
111
2
3
VIC: Mac Deployment Demo
Install Docker for Mac
Install VIC using provided installer
BEFORE starting VIC, copy your own data to VIC folder
Start the VIC application and perform the data Ingestion
11
Admin tools with different options to add/remove data
from the Image Classification engine
VIC: Advanced Deployment
Native deployment successfully tested on Ubuntu 14 LTS,macOs Sierra. Installation scripts can be provided.
Frontend consists of a Django Application and the imagedownloading engine (Google web engine only)
– Could provide support for other downloading engines
– Source code: https://gitlab.com/vgg/vgg_frontend
Backend consists of a python wrapper and a native imageclassification component
– Source code: https://gitlab.com/vgg/vgg_classifier
Backend and Frontend communicate via an API. Newbackends can be incorporated.
Docker deployment requires Docker v17.03.1-ce.12
VIC: Stats
Data ingestion:
– Dataset: ~14K images, ~27 min (multi-threaded)
– Negatives: ~17K images, ~1 hour (single-thread)
Backend data footprint
– Dataset: ~2.4 GB
– Extracted dataset features: 7.6 MB
– Negative images: ~416.7 MB
– Extracted negative features: 9.5 MB
Backend memory footprint: 185 MB (reducible with PQ)
One cached query: 18 MB (16 MB of training images)
Docker image: 1.4 GB13
Image Classification
Dr. Ernesto CotoResearch Software Engineer
Visual Geometry GroupDepartment of Engineering Science, University of Oxford