Data Mining and Neural Networks Danny Leung
-
Upload
abhinav-pandey -
Category
Documents
-
view
14 -
download
0
description
Transcript of Data Mining and Neural Networks Danny Leung
-
Data Mining and Neural Networks
Danny LeungCS157B, Spring 2006Professor Sin-Min Lee
-
Artificial Intelligence for Data MiningNeural networks are useful for data mining and decision-support applications.
People are good at generalizing from experience.
Computers excel at following explicit instructions over and over.
Neural networks bridge this gap by modeling, on a computer, the neural behavior of human brains.
-
Neural Network Characteristics
Neural networks are useful for pattern recognition or data classification, through a learning process.
Neural networks simulate biological systems, where learning involves adjustments to the synaptic connections between neurons
-
Anatomy of a Neural Network
Neural Networks map a set of input-nodes to a set of output-nodes
Number of inputs/outputs is variable
The Network itself is composed of an arbitrary number of nodes with an arbitrary topology
Neural Network
Input 0
Input 1
Input n
...
Output 0
Output 1
Output m
...
-
Biological BackgroundA neuron: many-inputs / one-output unit
Output can be excited or not excited
Incoming signals from other neurons determine if the neuron shall excite ("fire")
Output subject to attenuation in the synapses, which are junction parts of the neuron
-
Basics of a NodeA node is an element which performs a function
y = fH((wixi) + Wb)ConnectionNode
fH(x)
Input 0
Input 1
Input n
...
W0
W1
Wn
+
Output
+
...
Wb
-
A Simple PreceptronBinary logic application
fH(x) [linear threshold]
Wi = random(-1,1)
Y = u(W0X0 + W1X1 + Wb)
fH(x)
Input 0
Input 1
W0
W1
+
Output
Wb
-
Preceptron TrainingIts a single-unit network
Adjust weights based on a how well the current weights match an objective
Perceptron Learning Rule
Wi = * (D-Y).Ii
= Learning RateD = Desired Output
-
Neural Network LearningFrom experience: examples / training data
Strength of connection between the neurons is stored as a weight-value for the specific connection
Learning the solution to a problem = changing the connection weights
-
Neural Network LearningContinuous Learning Process
Evaluate output
Adapt weights
Take new inputs
Learning causes stable state of the weights
-
Learning PerformanceSupervisedNeed to be trained ahead of time with lots of data
Unsupervised networks adapt to the inputApplications in Clustering and reducing dimensionalityLearning may be very slowNo help from the outsideNo training data, no information available on the desired outputLearning by doingUsed to pick out structure in the input:ClusteringCompression
-
Topologies Back-Propogated NetworksInputs are put through a Hidden Layer before the output layer
All nodes connected between layers
Input 0
Input 1
Input n
...
Output 0
Output 1
Output o
...
O0
O1
Oo
H0
H1
Hm
...
...
Hidden Layer
-
BP Network Supervised Training
Desired output of the training examples
Error = difference between actual & desired output
Change weight relative to error size
Calculate output layer error , then propagate back to previous layer
Hidden weights updated
Improved performance
-
Neural Network Topology Characteristics
Set of inputs
Set of hidden nodes
Set of outputs
Increasing nodes makes network more difficult to train
-
Applications of Neural NetworksPrediction weather, stocks, disease
Classification financial risk assessment, image processing
Data association Text Recognition (OCR)
Data conceptualization Customer purchasing habits
Filtering Normalizing telephone signals (static)
-
OverviewAdvantagesAdapt to unknown situationsRobustness: fault tolerance due to network redundancyAutonomous learning and generalization
DisadvantagesNot exactLarge complexity of the network structure
-
Referenced WorkIntro to Neural Networks - Computer Vision Applications and Training Techniques. Doug Gray. www.soe.ucsc.edu/~taoswap/ GroupMeeting/NN_Doug_2004_12_1.ppt
Introduction to Artificial Neural Networks. Nicolas Galoppo von Borries. www.cs.unc.edu/~nico/courses/ comp290-58/nn-presentation/ann-intro.ppt