Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.
-
Upload
kelly-nash -
Category
Documents
-
view
215 -
download
1
Transcript of Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.
![Page 1: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/1.jpg)
ResearchResearch
Prediction Games in Infinitely Rich Worlds
Omid Madani
Yahoo! Research
![Page 2: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/2.jpg)
ResearchResearch
“Rather, the formation and use of categories is the stuff of experience.”
Philosophy in the Flesh, Lakoff and Johnson.
![Page 3: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/3.jpg)
ResearchResearch
Motivation
• Higher intelligence requires myriad inter-related categories
• How can such be acquired?• Programming them unlikely to be
successful:• Limits of our explicit knowledge• Unknown/unfamiliar domains• Making the system operational..
![Page 4: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/4.jpg)
ResearchResearch
Learn? … How?
• “Supervised” learning likely inadequate:• Required:
• ~millions of categories and beyond..• Billions of weights, and beyond..
• Inaccessible “knowledge” (see last slide!)
• Other approaches are fall short (incomplete, etc): clustering, RL, active learning, etc..
![Page 5: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/5.jpg)
ResearchResearch
This Work: An Exploration
• An avenue: “prediction games in infinitely rich worlds”
• Exciting part: • World provides unbounded learning
opportunity! (world is the teacher!)• World enjoys many regularities (e.g.
“hierarchical”)
![Page 6: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/6.jpg)
ResearchResearch
This Work• Describe the setting
• The games, categories, …
• Discuss:• Desiderata/constraints• Some of the many
challenges/problems
• Preliminary system/observations..
![Page 7: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/7.jpg)
ResearchResearch
The Game
• Repeat • Hide part(s) of the stream• Predict (use context)• Update• Move on
• Goal: predict better ... (subject to constraints)• In the process: categories at different levels of
abstraction learned• Some details: what parts to hide? How much
context? What order?
![Page 8: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/8.jpg)
ResearchResearch
In a Nutshell
Prediction System
…. 0011101110000….
After a While
predict observe & update
Prediction System
observe & updatepredict
low level categories
higher level categories(bigger chunks)(bits, characters, edges,…)
(e.g. words, digits, phrases, phone numbers, faces, visual objects, home pages, sites,…)
![Page 9: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/9.jpg)
ResearchResearch
Example of Games (text)
• .. d?an.. • System predictions (ranked or
assigned probabilities, or.. )• “r”• “e”• “o”• …
• I ? my bike to school.
![Page 10: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/10.jpg)
ResearchResearch
Categories
• Building blocks of intelligence?• Patterns that frequently occur
• External • Internal..• Useful for predicting other categories!• They can have structure/regularities
1. Composition (~conjunctions) of others
2. Grouping (~disjunctions)
![Page 11: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/11.jpg)
ResearchResearch
Categories
• Low level examples: 0 and 1 or characters• Provided to the system
• Higher levels:• Sequence of k bits• Words• Phrases• Regular expressions • Phone number, contact info, resume, ...
![Page 12: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/12.jpg)
ResearchResearch
Prediction Objective
• Desirable: learn higher level categories (bigger/abstract categories are useful externally)
• Question: how does this relate to improving predictions?
1. Higher level categories improve “context” and can save memory
2. Bigger, save time in playing the game (categories are atomic)
![Page 13: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/13.jpg)
ResearchResearch
Goal (evaluation criterion)
• Number of bits (characters) correctly predicted per
unit time (or per prediction action)
• Subject to constraints (space, time,..)
• How about entropy/perplexity? Categories are structured..
![Page 14: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/14.jpg)
ResearchResearch
Desiderata/Challenges/Issues• Lots of data!
• Efficiency: space and time!
• Noise:• Statistical insignificance• Significance, but for short time..
• Variety (need for abstraction)• Drift (e.g. developments within system)• Motivate: (primarily) online
algorithms/systems
![Page 15: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/15.jpg)
ResearchResearch
Desiderata/Challenges• Why need for “system”s?
• Multiple algorithms/parts needed• Persistence
• Long term learning: how can we make sure noise/errors do not accumulate?
• Control of the input stream..
![Page 16: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/16.jpg)
ResearchResearch
Why Now?
• Many category learning is possible/efficient!• Online• Noise tolerant
• Expectation: other problems are solvable..
![Page 17: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/17.jpg)
ResearchResearch
Preliminary Report
• Work in Progress!
• Plays the game in text
• Begins at character level
• No segmentation, just a stream
• Makes and predicts larger sequences (composition)
![Page 18: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/18.jpg)
ResearchResearch
Preliminary Observations
• Ran on Reuters RCV1 (text body) ( simply zcat dir/file* )
• 800k articles• >= 150 million learning/prediction episodes• Over 10 million categories built• 3-4 hours each pass
![Page 19: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/19.jpg)
ResearchResearch
Observations• Performance on held out (one of the
Reuters files):• 8-9 characters long to predict on average• Almost two characters correct on
average, per prediction action
• Can overfit/memorize! (long categories)
• Current: stop category generation in first pass
![Page 20: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/20.jpg)
ResearchResearch
![Page 21: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/21.jpg)
ResearchResearch
Current/Future
• Much work:• Learn groupings• Recognize/use “syntactic”
categories?• Prediction objective is ok?• Category generation.. What’s a good
method?
• Compare: language modeling, etc
![Page 22: Research Prediction Games in Infinitely Rich Worlds Omid Madani Yahoo! Research.](https://reader035.fdocuments.in/reader035/viewer/2022062720/56649f165503460f94c2c072/html5/thumbnails/22.jpg)
ResearchResearch
Much Related Work!
• Online learning, clustering, deep learning, Bayesian methods, hierarchical learning, importance of predictions (“On Intelligence”, “natural computations”), models of neocortex (“circuits of the mind”), concepts (“big book of concepts”), cumulative learning, neural nets, compression, learning an index of categories!