Multimedia Data Mining an Overview to Image Processing and Machine Learning by Zaheer Ahmad

8/7/2019 Multimedia Data Mining an Overview to Image Processing and Machine Learning by Zaheer Ahmad

http://slidepdf.com/reader/full/multimedia-data-mining-an-overview-to-image-processing-and-machine-learning 1/68

Multimedia Data Mining:An Overview to Image Processing

and

Machine Learning

Zaheer Ahmad

PhD Scholar

[email protected]

Department of Computer Science University of Peshawar

2/16/2011 1

mailto:[email protected]

mailto:[email protected]



Agenda

• Multimedia Data Mining

• Image Data Mining and Image Processing

•

Machine Learning• Learning Techniques and tools

• Neural Networks and its types

• Training (Learning) of Neural Network

2/16/2011 2



Multimedia Data mining

• Multimedia Data Mining is an interdisciplinaryand multidisciplinary field, used tointelligently retrieve and search multimedia

contents.

• A variety of techniques, from machine

learning, statistics, databases, knowledgeacquisition, data visualization, image analysis,high performance computing, and knowledge-based systems are used in MMM

2/16/2011 3



2/16/2011 4



MACHINE LEARNING

2/16/2011 5



Data for MMM

2/16/2011 6

Data a database ?

• No ----- mostly

• Web Image, Audio, Video

• Live Streaming• Geo Sensors data

• But yes….

•

video database• Image or audio database



• The word multimedia refers to a combination

of multiple media types together

• Multimedia Data Type

– Any Type of information medium that can be

represented, processed, stored and transmitted

over network in digital form

– Multi-lingual text, numeric, images, videos, audio,graphical, temporal, relational and categorical

data

2/16/2011 7



Definition

• MMM is a subfield of data mining that deals

with an extraction of implicit knowledge,

multimedia data relashionships, or other

patterns not explicitly stored in multimedia

databases

– Used for multimedia information system and

retrieval of content based image/audio/video andprovide search and efficient storage organization

2/16/2011 8



Media Types

• 0-dimensional data: This type of the data is the regular,

alphanumeric data. A typical example is the text data.

• 1-dimensional data: This type of the data has one dimension

of a space imposed into them. A typical example of this type

of the data is the audio data

• 2-dimensional data: This type of the data has two dimensions

of a space imposed into them. Imagery data and graphics data

are the two common examples of this type of data

• 3-dimensional data: This type of the data has three

dimensions of a space imposed into them. Video data and

animation data are the two common examples of this type of

data

2/16/2011 9



Multimeimedia Data

• Spatial Data – Generalize detailed geographic points into clusterd

regions, such as business, residential, industrial, oragricultural areas, according to land usage

•

Image Data – Size, color, shape, texture, orientation, and relative

postions and structure of the contained objects or regionsin the image

• Music data – Summarize its melody: based on the approximate pattern

that repeateldly occure in the segment

– Summarized its type: based on its tone, tempo, or themajor musical insturment played

2/16/2011 10



How Multimedia Data Mining System

Works

2/16/2011 11



Similarity Search in Multimedia data

• Description based retrieval systems

– Build indices and perform object retrieval based onimage descriptions, such as keywords, captions, sizeand time of creation

– Labor-intensive if performed manually

– Results are typically of poor quality if automated

• Content Based Retrieval Systems

• Support retrieval based on the image content,such as color, histogram, texture, shape, objectsand wavelet transforms

2/16/2011 12



Multidimensional Analysis of

Multimedia Data• Multimedia data Cube

– Design and construct similar to that traditional data cubes fromrelational data

– Contain additional dimensions and measures for multimediainformation such as color, texture, and shape

• The database doesn’t store images but their descriptors – Feature Descriptor: a set of vectors for each visual

characteristics• Color Vector: contains the color histogram

• MFC(Most Frequent Color) Vector: Five color centroids

• MFO(Most Frequent Orientation) Vector: Five edge orientationcentroid

– Layout Descriptor: Contains a color layout vector and an edgelayout vector

2/16/2011 13



Typical Architecture of MMM

2/16/2011 14



Image Data Mining

Image and Machine Learning

2/16/2011 15



What is an image?

• An image is a two dimensional

function, f(x,y), where x and y are

spatial coordinates, and the

amplitude of f at any pair of coordinates (x,y) is called the intensity

or grey level of the image at that

point.



17

Image Processing Stages

Image Acquisition

Image Processing

Image Segmentation

Image Analysis

Pattern Recognition

Analog to digital conversion

Remove noise,

improve contrast …

Find regions (objects)

in the image

Take measurements of

objects/relationships

Match the description with

similar description of known

objects (models)



19

Image Analysis

Input Image

Regions, objectsMeasurements

ImageAnalysis

Measurements:-Size-Position

-Orientation-Spatial relationship-Gray scale or color intensity



Image segmentation

The operation of distinguishing important objects from the

background (or from unimportant objects) based on different

feature of the image

Dark objects, bright background

Area B Area A



21

Image Segmentation

Input ImageRegions

Objects

Segmentation

-Clasify pixels into groups having similar characteristics

-Two techniques: Region segmentation—Color/smoothness

Edge detection



Histogram

The data contained in a digitalimage can be displayed as a

histogram which is a plot of the

pixel values ranging from black

to white versus the number of pixels that have that particular

value.

d h h d f



Edge through Gradient Information

Edge Location

Edge Direction i

),( ii y x

Neighborhood pixels

Sharpness Change / Contrast change



25

Pattern Recognition (PR)

- Measurements- Stuctural

descriptionsClass identifier

PatternRecognition

feature vector

set of information data



Content Based Image Retrieval

26



27

Fingerprint recognition system

Fingerprintsensor

Fingerprintsensor

Feature Extractor

Feature Extractor

Feature Matcher

ID

Enrollment

Identification

Templatedatabase



Machine Learning

A computer program is said to learn from

experience ‘E’ with respect to some class of

tasks ‘T’ and performance measure ‘P’,

If its

performance at tasks in T, as measured by P,

improves with experience E.

Mitchell (1997):

2/16/2011 28



Machine Learning

Things learn when they change their behavior in

a way that makes them perform better in the

future.

From Witten and Frank (2000)

2/16/2011 29



Machine Learning

• ML is a scientific discipline that is concerned

with the design and development of algorithms

that allow computers to evolve behaviors based

on empirical data, such as from sensor data or

databases.• A major focus of machine learning research is to

automatically learn to recognize complex

patterns and make intelligent decisions based

on data.

2/16/2011 30



• the difficulty lies in the fact that the set of all

possible behaviors given all possible inputs is

too large to be covered by the set of observed

examples (training data).

• Hence the learner must generalize from the

given examples, so as to be able to produce a

useful output in new cases

2/16/2011 31



Types of Learning

2/16/2011 32

• Supervised Learning

Learning a mapping between an input x and

a desired output y

• Unsupervised LearningUnderstanding the relationships between

data components

• Reinforcement Learning

Learning to act in the environment based on

the delayed rewards

Cl f L i



Classes of Learning

Machine learning is not only about classification.

Classification learning: learn to put instances intopre-defined classes-----competitive network:

selects one unit in the output layer (target class)---

(Supervised Learning)

Association learning: learn relationships between theAttributes------ new response becomes associated

with a particular stimulus ---pattern associator:

recalls input patterns based on similarity

Clustering: discover classes of instances that belong

Together------- (Unsupervised)self-organizing map

(SOMs)

2/16/2011 33



Learning Tools and Techniques

inShort

2/16/2011 34



Learning Rules

• if outlook = sunny and humidity = high then play= no

• if outlook = rainy and windy = true then play = no

• if outlook = overcast then play = yes• if humidity = normal then play = yes

• if none of the above then play = yes

BEST But LABOURUS , HARD TO CODE AND COVERin Large Domains

2/16/2011 35



Learning Decision Trees

• Example: XOR (familiar from connectionist

networks).

Nodes represent decisions on attributes, leaves represent classifications.

Some how like Learning Rules

2/16/2011 36



Principal component analysis

• PCA is applied as a data reduction or structuredetection method

• combining two correlated variables into onefactor

• PCA defined as an orthogonal lineartransformation that transforms the data to a newcoordinate system such that the greatest varianceby any projection of the data comes to lie on the

first coordinate (called the first principalcomponent), the second greatest variance on thesecond coordinate

2/16/2011 37



Support Vector Machine

• Support Vector Machine is a classifier derivedfrom statistical learning theory by VladimirVapnik and his co-workers

• Used for large data set• Good for text classification

• Work as multilayer perceptron

2/16/2011 38



Hidden Markov Model

2/16/2011 39



Genetic Algorithms

2/16/2011 40



Neural Networks

41



Inputs Outputs

Connection between cells

NN A Brain-Inspired Model

in

out

42



Physical Structure of biological

neuron

2/16/2011 43

• Nerve cells are main processing element in our

central nervous system.

• Humans generally have about 100 billion nerve

cells in the entire nervous system.• Axon and dandroid are signal carrier away and

toward cell body respectively

• Synapse is the point at which the axon of one cell

interconnects with a dendrite of another cell• A basic nerve cell is thought as a black box



NN A Brain-Inspired Model

• A neural network acquires knowledge through

learning.

• A neural network's knowledge is stored within

inter-neuron connection strengths known as

synaptic weights.

• The largest modern neural networks

achieve the complexity comparable to anervous system of a fly.

44



Historical Background

• 1943 McCulloch and Pitts proposed the firstcomputational models of neuron.

• 1949 Hebb proposed the first learning rule.

• 1958 Rosenblatt’s work in perceptrons.• 1969 Minsky and Papert’s exposed limitation of the

theory.

• 1970s Decade of dormancy for neural networks.

• 1980-90s Neural network return (self-organization,back-propagation algorithms, etc)

45



NN Applications

• Process Modeling and Control- Creating a neural network model for a physical

plant then using that model to determine the best control settings for the plant.

• Machine Diagnosis- Detect when a machine has failed so that the system canautomatically shut down the machine when this occurs.

• Target Recognition- Military application which uses video and/or infrared image data todetermine if an enemy target is present.

• Medical Diagnosis- Assisting doctors with their diagnosis by analyzing the reportedsymptoms and/or image data such as MRIs or X-rays.

• Target Marketing- Finding the set of demographics which have the highest responserate for a particular marketing campaign.

• Voice Recogntion- Transcribing spoken words into ASCII text.

• Financial Forecasting( Stock predication) - Using the historical data of a security to

predict the future movement of that security.• Quality Control - Attaching a camera or sensor to the end of a production process to

automatically inspect for defects.

• Intelligent Search - An internet search engine that provides the most relevant contentand banner ads based on the users' past behavior.

• Fraud Detection - Detect fraudulent credit card transactions and automatically decline

the charge. 46

http://www.nd.com/apps/science.html



How NN Work ( Mathematically)

• Linear and Non Linear Pattern / Classification• Regression / Function Estimation

• Curve Fitting

Why to USE NN

• Parallel Processing

•

Fault tolerance• Self-organization

• Generalization ability

• Continuous adaptivity47



48

Artificial Neurons

• Neural networks are made up of nodes which have – Input edges, each with some weight

– Output edges (with weights)

– An activation level (a function of the inputs)

• Weights of edges can be positive or negative and may change

over time (learning)

• The output function is the weighted sum of the activation levels

of inputs

• The activation level is a linear or non-linear transfer function “a”

of the input :

• Some nodes are inputs, some are outputs.

A ifi i l N l N k



Artificial Neural Networks

Block Diagram

2/16/2011 49

A ifi i l N l N k



Artificial Neural Networks

Process

2/16/2011 50

The Perceptron



The Perceptron

51

x1

x2

xn

.

.

.

w1

w2

wn

wn+1

Biasxn+1=-1

a= bias+wi xi

y

1 if a 0y= 0 if a <0{

q=wn+1

•Bias , the extra weight connected to a constant is called the bias of

the element

• It enables to set the threshold equal to zero which help in

calculation•To get an extra dimension for representation This means

that every point in (n + 1)-dimensional weight space can be

associated with a hyperplane in (n + 1)-dimensional extended input

space.



Logical Operations

2/16/2011 52

Threshold= 2

Threshold= 2



2/16/2011 53

Threshold= 2

The first layer performs the two AND NOT's and the

second layer performs the OR. Both Z neurons and

the Y neuron have a threshold of 2

X1 XOR X2 = (X1 AND NOT X2) OR (X2 AND NOT

X1)



Linear Separability Problem

• If two classes of patterns can be separated by a decision boundary,

represented by the linear equation

then they are said to be linearly separable. The simple network can

correctly classify any patterns.

• Decision boundary of linearly separable classes can be determinedeither by some learning procedures or by solving linear equation

systems based on representative patterns of each classes

• If such a decision boundary does not exist, then the two classes are

said to be linearly inseparable.• Linearly inseparable problems cannot be solved by the simple

network , more sophisticated architecture is needed.

01

n

i iiw x b

54



• Examples of linearly separable classes

- Logical AND function

patterns (bipolar) decision boundary

x1 x2 y w1 = 1-1 -1 -1 w2 = 1-1 1 -1 b = -11 -1 -1 q = 01 1 1 -1 + x1 + x2 = 0

- Logical OR function


x1 x2 y w1 = 1-1 -1 -1 w2 = 1-1 1 1 b = 11 -1 1 q = 01 1 1 1 + x1 + x2 = 0

x

oo

o

x: class I (y = 1)o: class II (y = -1)

x

xo

x


55

Equation of Line ( Decision Boundary )



• Examples of linearly inseparable classes

- Logical XOR (exclusive OR) function


x1 x2 y-1 -1 -1-1 1 11 -1 11 1 -1

o

xo

x


56



Multilayer NN

• Neural Net for Nonlinear Classification

• Combination of Perceptron

• Back propagation learning

57



What do each of the layers do?

1st layer draws

linear boundaries

2nd layer combines

the boundaries

3rd layer can generate

arbitrarily complex boundaries

Multilayer FFNN

A NN with one or more than one hidden layers

58

B k ti Al ith



Back propagation Algorithm• Multiple outputs.

• Forward pass:• Error calculation:

• Backward propagation:

• No guarantee to in getting best possibleweights after correcting.

• Classifies inputs into multiple classes.

• Can be modified to represent any function.



NN Training Data

• Training Set: this data set is used to adjust the weights on theneural network.

• Validation Set: this data set is used to minimize overfitting. – not adjusting the weights of the network with this data set,

– just verifying that any increase in accuracy over the training data set

actually yields an increase in accuracy over a data set that has notbeen shown to the network before, or at least the network hasn'ttrained on it (i.e. validation data set).

– If the accuracy over the training data set increases, but the accuracyover then validation data set stays the same or decreases,

– then you're overfitting your neural network and you should stop

training.• Testing Set: this data set is used only for testing the final solution in

order to confirm the actual predictive power of the network.

2/16/2011 60

N d A i i F i



Neuron and Activation Functions

2/16/2011 61

A i i F i



Activation Functions

2/16/2011 62

These functions can be defined as follows.

Stept(x) = 1 if x >= t, else 0Sign(x) = +1 if x >= 0, else -1

Sigmoid(x) = 1/(1+e-x)

Selection of Nodes for



Selection of Nodes for

Neural Network

• Input Nodes----Image/data size

• Output node---output binary

• Middle Layer----o ooo oo….

– Keep middle layer smaller to Generalize and not

memorize

2/16/2011 63

Perceptron Learning Algorithm:



64


Initialise weights and threshold.

Set w i (t), (0 <= i <= n), to be the weight i at

time t , and ø to be the threshold value in the

output node. Set w 0

to be -ø, the bias, and x 0

to be always 1.

Set w i ( 0 ) to small random values, thus

initialising the weights and threshold.

Present input and desired outputPresent input x 0, x 1, x 2, ..., x n and desired

output d(t)

Calculate the actual output

y(t) = f h[w 0(t)x 0(t) + w 1(t)x 1(t) + .... + w n(t)x n(t)]

Adapts weights

w i (t+1 ) = w i (t) + ñ[d(t) - y(t)]x i (t) , where 0 <= ñ <= 1 is a positive gain function that controls

the adaption rate.

Steps iii. and iv. are repeated until the iteration

error is less than a user-specified error

threshold or a predetermined number of

iterations have been completed.


start: The weight vector w0 is

generated randomly,set t := 0

test: A vector x 2 P [ N is selected

randomly,

if x 2 P and wt · x > 0 go to test,

if x 2 P and wt · x 0 go to add,

if x 2 N and wt · x < 0 go to test,

if x 2 N and wt · x 0 go to subtract.

add: set wt+1 = wt + x and t := t +

1, goto test

subtract: set wt+1 = wt − x and t :=

t + 1, goto test

Neural Networks



65

Neural Networks –

Training

Backpropagation training cycle



Urdu OCR Input Data Examplefeeded to FNN

66



2/16/2011 67



ouY

68

Thank



References

• Data Mining and Knowledge Discovery Series, Chapman &Hall/CRC

• Neural Networks a Systematic Approach

• Matlab - development of neural network theory for artificial

life-thesis, matlab and java code

• Digital Image Processing By Gonzalez Using Matlab

• Wikiiiiiiiiiiiiiiiiiiiiiipedia

• And…….

Multimedia Data Mining an Overview to Image Processing and Machine Learning by Zaheer Ahmad

Documents

Transcript of Multimedia Data Mining an Overview to Image Processing and Machine Learning by Zaheer Ahmad