Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge...
-
date post
21-Dec-2015 -
Category
Documents
-
view
218 -
download
0
Transcript of Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge...
![Page 1: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/1.jpg)
Machine Learning
CSE 473
![Page 2: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/2.jpg)
© Daniel S. Weld 2
473 Topics
Agency
Problem Spaces
Search
Knowledge Representation
Reinforcement
Learning Inference Planning
SupervisedLearning
Logic Probability
![Page 3: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/3.jpg)
© Daniel S. Weld 3
Machine Learning Outline
• Machine learning: What & why? Bias
• Supervised learning Classifiers
A supervised learning technique in depth Induction of Decision Trees
• Ensembles of classifiers• Overfitting
![Page 4: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/4.jpg)
© Daniel S. Weld 4
Why Machine Learning
•Flood of dataWalMart – 25 TerabytesWWW – 1,000 Terabytes
•Speed of computer vs. %#@! of programmingHighly complex systems (telephone switching systems)Productivity = 1 line code @ day @ programmer
•Desire for customizationA browser that browses by itself?
•Hallmark of IntelligenceHow do children learn language?
![Page 5: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/5.jpg)
© Daniel S. Weld 5
Applications of ML
• Credit card fraud• Product placement / consumer behavior• Recommender systems• Speech recognition
Most mature & successful area of AI
![Page 6: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/6.jpg)
© Daniel S. Weld 6
Examples of Learning
• Baby touches stove, gets burned, next time…
• Medical student is shown cases of people with disease X, learns which symptoms…
• How many groups of dots?
![Page 7: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/7.jpg)
© Daniel S. Weld 7
What is Machine Learning??
![Page 8: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/8.jpg)
© Daniel S. Weld 8
Defining a Learning Problem
• Task T: Playing checkers
• Performance Measure P: Percent of games won against opponents
• Experience E: Playing practice games against itself
A program is said to learn from experience E with respect to task T and performance measure P, if it’s performance at tasks in T, as measured by P, improves with experience E.
![Page 9: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/9.jpg)
© Daniel S. Weld 9
Last Class?
• Task T: Programming by demonstration
• Performance Measure P:
• Experience E:
A program is said to learn from experience E with respect to task T and performance measure P, if it’s performance at tasks in T, as measured by P, improves with experience E.
![Page 10: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/10.jpg)
© Daniel S. Weld 10
Issues
• What feedback (experience) is available?• How should these features be represented?• What kind of knowledge is being increased? • How is that knowledge represented?• What prior information is available? • What is the right learning algorithm?• How avoid overfitting?
![Page 11: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/11.jpg)
© Daniel S. Weld 11
Choosing the Training Experience• Credit assignment problem:
Direct training examples: • E.g. individual checker boards + correct move for
each• Supervised learning
Indirect training examples : • E.g. complete sequence of moves and final result• Reinforcement learning
• Which examples: Random, teacher chooses, learner chooses
![Page 12: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/12.jpg)
© Daniel S. Weld 12
Choosing the Target Function
• What type of knowledge will be learned?• How will the knowledge be used by the
performance program?• E.g. checkers program
Assume it knows legal moves Needs to choose best move So learn function: F: Boards -> Moves
• hard to learn Alternative: F: Boards -> R
Note similarity to choice of problem space
![Page 13: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/13.jpg)
© Daniel S. Weld 13
The Ideal Evaluation Function
• V(b) = 100 if b is a final, won board • V(b) = -100 if b is a final, lost board• V(b) = 0 if b is a final, drawn board• Otherwise, if b is not final
V(b) = V(s) where s is best, reachable final board
Nonoperational…Want operational approximation of V: V
![Page 14: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/14.jpg)
© Daniel S. Weld 14
How Represent Target Function
• x1 = number of black pieces on the board• x2 = number of red pieces on the board• x3 = number of black kings on the board• x4 = number of red kings on the board• x5 = num of black pieces threatened by red• x6 = num of red pieces threatened by black
V(b) = a + bx1 + cx2 + dx3 + ex4 + fx5 + gx6
Now just need to learn 7 numbers!
![Page 15: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/15.jpg)
© Daniel S. Weld 15
Example: Checkers• Task T:
Playing checkers• Performance Measure P:
Percent of games won against opponents• Experience E:
Playing practice games against itself• Target Function
V: board -> R• Representation of approx. of target function
V(b) = a + bx1 + cx2 + dx3 + ex4 + fx5 + gx6
![Page 16: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/16.jpg)
© Daniel S. Weld 16
Target Function
• Profound Formulation: Can express any type of inductive
learning as approximating a function• E.g., Checkers
V: boards -> evaluation • E.g., Handwriting recognition
V: image -> word• E.g., Mushrooms
V: mushroom-attributes -> {E, P}
![Page 17: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/17.jpg)
© Daniel S. Weld 17
More Examples
![Page 18: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/18.jpg)
© Daniel S. Weld 18
More Examples
• Collaborative Filtering Eg, when you look at book B in Amazon It says “Buy B and also book C together &
save!”
• Automatic Steering
![Page 19: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/19.jpg)
© Daniel S. Weld 19
Supervised Learning
• Inductive learning or “Prediction”: Given examples of a function (X, F(X)) Predict function F(X) for new examples X
• Classification F(X) = Discrete
• Regression F(X) = Continuous
• Probability estimation F(X) = Probability(X):
TaskPerformance Measure Experience
![Page 20: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/20.jpg)
© Daniel S. Weld 20
Why is Learning Possible?
Experience alone never justifies any conclusion about any unseen instance.
Learning occurs when PREJUDICE meets DATA!
Learning a “FOO”
![Page 21: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/21.jpg)
© Daniel S. Weld 21
Bias•The nice word for prejudice is “bias”.•What kind of hypotheses will you consider?
What is allowable range of functions you use when approximating?
•What kind of hypotheses do you prefer?
![Page 22: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/22.jpg)
© Daniel S. Weld 22
Some Typical BiasThe world is simple
Occam’s razor“It is needless to do more when less will
suffice” – William of Occam,
died 1349 of the Black plague
MDL – Minimum description length Concepts can be approximated by ... conjunctions of predicates
... by linear functions
... by short decision trees
![Page 23: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/23.jpg)
© Daniel S. Weld 23
![Page 24: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/24.jpg)
© Daniel S. Weld 24
![Page 25: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/25.jpg)
© Daniel S. Weld 25
![Page 26: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/26.jpg)
© Daniel S. Weld 26
![Page 27: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/27.jpg)
© Daniel S. Weld 27
Two Strategies for ML
•Restriction bias: use prior knowledge to specify a restricted hypothesis space.
Version space algorithm over conjunctions.•Preference bias: use a broad hypothesis space, but impose an ordering on the hypotheses.
Decision trees.
![Page 28: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/28.jpg)
© Daniel S. Weld 28
![Page 29: Machine Learning CSE 473. © Daniel S. Weld 2 473 Topics Agency Problem Spaces Search Knowledge Representation Reinforcement Learning InferencePlanning.](https://reader036.fdocuments.in/reader036/viewer/2022062421/56649d555503460f94a333f0/html5/thumbnails/29.jpg)
© Daniel S. Weld 29