Neural Networks and Their Applications John Paxton Montana State University August 14, 2003.
-
date post
20-Dec-2015 -
Category
Documents
-
view
217 -
download
0
Transcript of Neural Networks and Their Applications John Paxton Montana State University August 14, 2003.
Problem Domains
• Storing and recalling patterns
• Classifying patterns
• Mapping inputs onto outputs
• Grouping similar patterns
• Finding solutions to constrained optimization problems
Comparison
Biological Neural Net Artificial Neural Net
Soma Neuron
Dendrite Input
Axon Output
Synapse Weight
Neuron Output
• X = ( ∑ xiwi )
• y = fn ( X )
• Common functions (fn)– sign function (-1 if X < 0, else 1)– sigmoid function (1 / (1 + e-x) )
Perceptron
• Can automatically learn the weights in a provable fashion!
• But can only learn linearly separable concepts.
no
yes
yes
no
Multilayer Neural Network
• Can include zero or more hidden layers
• Has a learning algorithm (backpropagation) that works in practice!
XOR
Backpropagation
• Determine network topology
• Initialize weights
• Present a training example
• Apply inputs, calculate activations in middle layer, then calculate activations in output layer
• Calculate errors in output layer
• Calculate errors in hidden layer
Backpropagation
• Update weights
• Repeat process until some stopping condition is met
• Possible stopping conditions– largest error is below some threshold– total error is no longer decreasing– a time limit is exceeded
Strengths
• Very versatile. They can predict, they can classify, they can cluster.
• They produce good results in complicated domains.
• Can handle categorical and continuous data.
• Available in many off-the-shelf products
Weaknesses
• All inputs must be massaged onto the range [-1 .. 1 ].
• Can not explain results.
• May converge on an inferior solution.
• Determining the topology is as much an art as it is a science.
• Might take a long time to converge.
Commercial Applications
• Neural networks were involved in more than 1 billion U.S. dollars in 1997!
Business
• Marketing– Microsoft. Direct mail marketing.– Albertsons. Determine the connection
between buying diapers and buying beer.– BehavHeuristics Inc. Forecasts demand of
airline flights.
• Real Estate– HNC. Automated Real Estate Appraisal.
Document and Form Processing
• Machine Printed Form Processing– Caere Corporation. Optical character
recognition (FoxMaster).– Synaptics. Check reader.
• Hand Written Character Recognition.– Eastman Kodak. Forms processing for UK
motor vehicle registration.– Fujitsu. Input to pen based computers.
Document and Form Processing
• Cursive Handwriting Character Recognition– Apple Newton 120. Input to PDA.
• Graphic Recognition– Fein-Marquet Associates, Inc. Converts a
hand drawn chemistry picture into a table.
Food Industry
• Odor/Aroma Analysis– Sharp. Cooking control via an electronic nose
in a microwave oven.
• Produce Development– M&M/Mars. Improved chemical formulations
of products.
Food Industry
• Quality Assurance– Anheuser-Busch. Beer testing.– Florida Department of Citrus. Pulp wash
detection.– Frito-Lay. Potato chip testing.
Financial Industry
• Market Trading– Gerber Baby Foods. Cattle futures trading.– John Deere. Pension management.– Walkrich Investment Advisors. Stock
valuation.
• Credit Rating– Chase Financial. Forecast credit worthiness.
Financial Industry
• Fraud Detection– Dunn and Bradstreet. Check approval.– HNC. Credit card fraud detection (Falcon).– Mastercard. Deviations in spending habits.
Energy
• Electrical Load Forecasting– Bayernwerk AG.
• Hydroelectric Dam Operation– Tauernkraftwerke. Dam displacement
prediction.
• Natural Gas– Northern Natural Gas. Predict gas index
prices.
Manufacturing
• Process Controllers– Nippon Steel. Continuous casting.– Siemans. Rolling mill.
• Quality Control Systems– Dunlop. Tires.– Intel. Computer chips.– Volvo. Diesel knock testing. Paint inspection.
Medical and Health Care
• Image Analysis– NeuroMedical Systems, Inc. Pap smears.
• Drug Development– Vysis Inc. Protein analysis
• Resource Allocation– Anderson Memorial Hospital. Predict use of
hospital resources.
Science and Engineering
• Chemical Engineering– StellarNet Inc. Spectroscopy.
• Electrical Engineering– NeuroCad Inc. Optimized circuit routing.
• Weather– National Weather Service.
Transportation and Communication
• Transportation– London Underground. Fault detection.– Rolls Royce. Fault detection.
• Communication– AT&T/Lucent. Echo cancellation systems
(more than 20 years).