Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our...
-
Upload
alex-acm-sc-library -
Category
Documents
-
view
221 -
download
0
Transcript of Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our...
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
1/69
Artificial Intelligence
Our Ultimate Dream
Mohammad Abdelaziz Gowayyed
May 2013
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
2/69
Agenda
Introduction
Example: How to build a machine to play
chess?
Artificial Intelligence Subfields
Feature Extraction for Classification
Artificial Neural Networks
Future of Artificial Intelligence
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
3/69
What is Artificial Intelligence?
The study of computer systems that attempt
to model and apply the intelligence of the
human mind
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
4/69
Artificial Intelligence Directions
Build Systems that:
act like humans
act rationally
think like humans
think rationally
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
5/69
Act like human: The Turing Test
Turing Test:
If a man had a conversation with a machine and
he couldnt tell, whether it was a machine or a
human, this machine passes the Turing Test
It means that the machines perfectly acts like a
human
Does it need to think like a human to act like ahuman?
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
6/69
Think like human: Cognitive Science
We must somehow determine first how
human think, do we know?
Introspection
Psychological experiments
Cognitive Science: construct precise and
testable theories of the workings of humanminds through computer models and
psychological experiments.
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
7/69
Think rationally: Laws of Thought
Formalizing logic
Main emphasis is on inference
Example: Socrates is a man; all men are
mortal; therefore Socrates is mortal.
By 1965, programs existed that could, givenenough time and memory, take a description
of a problem in logical notation and find thesolution to the problem, if one exists.
Problems?
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
8/69
Act rationally: The rational agent
Rational behavior: doing the right thing
The right thing: that which is expected to
maximize goal achievement, given the
available information
Take decisions
Reinforcement Learning: learn fromenvironment feedback based on my choices
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
9/69
Is there a benchmark for humanintelligence?
May be Chess?
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
10/69
How to program a machine to play
Chess?
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
11/69
How to program a machine to play
Tic Tac Toe?
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
12/69
Easy?
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
13/69
What is the difference betweenChess and Tic Tac Toe?
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
14/69
The general idea is to build a treerepresenting different possible paths,
each path represents sequence of
choices from you and your opponent.The tree will help the computer
choose
That is called aGame Tree
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
15/69
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
16/69
Min-Max Algorithm
Two agents are playing
Zero-Sum Game
Any loss to you is an equal win for your opponent
Assume that you are playing with a rational
player; he will try to maximize his profit
At each step, you want to maximize your profitand assume that the opponent will minimize it
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
17/69
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
18/69
Min-Max Algorithm
Is it feasible to create the whole tree? Is it finite?
What about the depth?
Will we be forced to choose random moves?
How to evaluate each state? Heuristic function (e.g. number of safe pieces)
Different algorithms for traversing the tree Depth first
Breadth First A*
Do we need to visit the whole tree? Alpha-Beta pruning
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
19/69
Heuristic Function
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
20/69
Can we say that the computer is
now intelligent because it candefeat human in chess?
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
21/69
There are still a lot of tasks that
human does easily while it is very
hard for machines
Can you name a few?
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
22/69
Chinese Room
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
23/69
Chinese Room
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
24/69
Machine vs. Human
Machine
Easy Tasks
Multiply 4878 * 1254
Play chess Hard Tasks
Understand concepts
Recognize faces
Recognize voices
Human
Easy Tasks
Recognize faces
Recognize voices Understand concepts
Hard Tasks
Multiply 4878 * 1254
Play chess
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
25/69
Some Artificial Intelligence Subfields
Machine Learning
Let machines learn rules (e.g. to classify or tochoose between alternatives Reinforcement
Learning) Computer Vision
Teach machines how to recognize visual world
Natural Language Processing
Teach machines how to recognize humanlanguages
Speech Recognition
Teach machines how to recognize human voices
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
26/69
Some Computer Vision Tasks
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
27/69
Face Detection
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
28/69
Image Classification
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
29/69
Human Action Recognition
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
30/69
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
31/69
Feature Representation
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
32/69
Feature Representation
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
33/69
How is computer perception done?
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
34/69
Feature Representation
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
35/69
Computer Vision Features
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
36/69
Audio Features
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
37/69
NLP Features
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
38/69
Sensor Representation in Human Brain
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
39/69
Can we mimic what already happens in
human brains?
The human brain consists of a network of
neurons.
Human brains learns to recognize images by
the same method as it learns to recognize
voices.
Scientists still dont know how the brain
works.
Scientists only know some concepts that brain
applies (e.g. sparse and general
representations)
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
40/69
Human Nervous System
Around 85,000,000,000neurons each one
connected to around 1000 others
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
41/69
Number of neurons in elephant brains
11,000,000,000 neurons
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
42/69
Number of neurons in horses brains
1,200,000,000 neurons
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
43/69
Neuron
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
44/69
Neuron
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
45/69
Number of neurons in human brains
From 19,000,000,000 to
23,000,000,000 neurons
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
46/69
Artificial Neural Networks
We know that human brains consists of hugeneural network, lets study simpler artificialneural networks.
And some day, we may be able to createartificial neural network that works the sameas the real neural network works.
Some people thought that it will never work!They had their arguments.
Thats why some researchers dont like neural
networks.
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
47/69
Neural Networks
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
48/69
Neural Networks
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
49/69
Backpropagation
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
50/69
Backpropagation
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
51/69
How to build better Neural Networks? Increase Layers?
How would we train them?
The network with more than 1 hidden layer is
called a deep network
Training a deep network is not easy using
Backpropagation.
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
52/69
The birth of Deep Learning Until 2006, Support
Vector Machines always
outperformed neural
networks on benchmarkclassification tasks.
At 2006, Hinton started
the new revolution on
machine learning: Deep
Learning
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
53/69
The birth of Deep Learning At 2006, Hinton proposed a greedy layer-wise
algorithm for training a deep belief network.
That was the first successful learning
algorithm for a deep network. Achieved best accuracy on MNIST hand
written digits dataset.
Restricted Boltzmann Machines. The network works also as a generative model
to generate data.
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
54/69
Deep Belief Network
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
55/69
Demo
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
56/69
Classical Approach Train a classifier using labeled data; then test
the classification accuracy on another testing
data.
Row data can be: images, videos, audio,
natural language
We start with extracting appropriate features.
Hand-crafted feature extraction will differ
according to the nature of the data.
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
57/69
Deep Learning Approach Pre-train the network using non-labeled data-
available.
Fine-tune the weights using some labeled
data. Also called: Unsupervised Feature Learning
Deep Learning Models:
Deep Belief Networks Stacked Autoencoders
Convolutional Neural Networks
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
58/69
Sparse Autoencoders
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
59/69
What each neuron learnt?
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
60/69
What each neuron learnt?
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
61/69
Motivating Results Unsupervised feature extraction
outperformed hand-crafted features in set of
popular machine recognition tasks:
Human Activity Recognition
Handwritten digits Recognition
Audio classification
ImageNet classification
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
62/69
Biological Mapping Interesting paper at 2005: Invariant visual
representation by single neurons in the
human brain
They showed that some neurons respond with
high selectivity to, for instance, images, sound
and name of the actress Halle Berry!
How Many Computers to Identify a
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
63/69
How Many Computers to Identify a
Cat? 16,000
Andrew NG, with Google at 2012, trained a
very large network: on16,000 machines.
They trained the network on random
unlabeled videos from Youtube, without any
guidance.
They found that one neuron of the network
learnt to detect human faces and another onelearnt to detect cat faces!
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
64/69
Some neurons learnt human face!
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
65/69
Other neurons learnt cat faces!
How Many Computers to Identify a
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
66/69
How Many Computers to Identify a
Cat? 16,000
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
67/69
Future of Artificial Intelligence
With the increase of computational power,
every day, scientists are able to build more
complex neural networks
Challenges
Network topologies
Learning algorithms
Neuroscientists can lead an AI revolution if
they came closer to how the brain works
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
68/69
Acknowledgment
Some slides are taken from Andrew Ng deep
learning tutorial.
Images used here are for educational
purposes, any further use should return back
to the original copyright holders.
-
7/30/2019 Alex ACM SC Machine Learning Day [Materials] | Machine Learning& Artificial Intelligence ... Our Ultimate Dream. By Eng. Mohamad Abdelaziz Gowayed
69/69
Thanks You
Questions?