Incremental Learning

Chris Mesterharm

Fordham University

Artificial Intellegence

• Make machines as smart or smarter than humans

• Intelligent machines can automate many tasks performed by humans

• Research started at beginning of modern computer era

Programming AI

• Difficult to program machines to be intelligent

• Some success with well structured problems

• Less success with “simpler” skills such as perception and vision

Machine Learning

• Teach instead of program machines

• Similar to humans and animals

But how do we model learning?

Induction

• The sun has risen every day in the past

• The sun will rise tomorrow

Learning from examples

Induction Examples

• Predict promoter sequences in DNA that specify where a gene is starting

• Predict the amount of change in the price of a particular stock in the market

• Predict whether or not an image contains a picture of a bicycle

• Predict tomorrow’s weather

Stock Market Example

• Goal is to predict how price of IBM stock will change tomorrow

• Learning algorithm looks for a prediction rule using any information that might be relevant– Previous prices and sales volume

– Economic indicators and index fund prices

– Analysts predictions and recent new reports

• All the information is collected together as an instance of the learning problem.

Stock Market Example

• Goal is to label a new instance with a price that is close to the future price

• We already have many labeled instances from previous days

• We use the previously labeled instances to learn a rule that works well on future instances

Batch LearningInstance label

Algorithm rule

ruleInstance label

Incremental Learning

• Assume you get feedback on your predictions

• Refine your rule with label feedback

• Allows the rule to adjust to changing conditions

Online Learning

• Algorithm has no instances at start

• Algorithm gets a single instance each trial

• A trial is composed of three steps– Get instance– Predict label– Discover true label and use that information to

refine the prediction rule

Online Learning

ruleInstance Predicted label

Algorithm True label

Online Learning Examples

• Financial market• Weather prediction• Sports• Predicting the future• Spam detection• Cooking

WinnowLearns a hyperplane to separate instances

x

x

x

x

xx

x

o

o

o

o

o

o

o

oo

o

x

x

x

Winnow

• Strong theoretical guarantees

• Performs well even with large instances

• Robust to noise• Can learn a changing

hyperplane

Acquisitions: Reuters

Tracking Winnow Example

Expanding and Testing

• Algorithm should allow– Delays in label feedback– New ways to use information in instances– Ability to internally generate labels

• Test online algorithms on– Financial problems– Robotics problems

Conclusion

• Online learning is a theoretically strong model for learning

• Online learning allows an incremental and adaptive style of learning that might be closer to the way humans and animals learn

• Winnow is a successful example of online learning that works well in practice