Integration of Search and Learning Algorithms

Eugene Fink

Basic intuition

An intelligent system should select or invent an effective approach to a given problem.

Challenges

• Select among available search algorithms (or invent a new algorithm)

• Restate the problem, to improve the efficiency of the selected algorithm

Search algorithm

searchproblem

Search algorithm

searchproblem solution ?

Search algorithm

searchproblem solution ?

A search module inputs a problem and runs until finding a solution, failing, or reaching a time limit.

Speed-up learning algorithm

speed-uplearning

problem

Speed-up learning algorithm

speed-uplearning

problemproblem and

new knowledge

Speed-up learning algorithm

speed-uplearning

problemproblem and

new knowledgesearch

Speed-up learning algorithm

speed-uplearning

problem

An algorithm inputs a problem and generates additional knowledge, which improves search efficiency.

problem andnew knowledge

search

… or static-analysis algorithm

staticanalysis

problem

An algorithm inputs a problem and generates additional knowledge, which improves search efficiency.

problem andnew knowledge

search

Goals• Construct an architecture for integration of

multiple search and learning modules• Centralized architecture• Integration tools

Goals• Construct an architecture for integration of

multiple search and learning modules

• Develop a control mechanism that chooses appropriate modules for each problem

• Centralized architecture• Integration tools

• Top-level control decisions• Reuse of accumulated knowledge

Related work

• AI systems with multiple learning and search modules: Soar, Prodigy, …

• Integration tools: Multiagent Planning Architecture (Wilkins and Myers), Plan++ (Yang et al.)

• Selection among available algorithms: Horvitz, Breese, Russell, Minton, ...

Shaper system

Three main parts:

• Library of search modules

• Library of learning modules

• Top-level control mechanism

System outline

System outline

System outline

Control center

System outline

Manualcontrol

Automaticcontrol

Control center

System outline

Manualcontrol

Automaticcontrol

Control center

Given a new problem:• Select appropriate modules• Apply them to solve the problem• Repeat if necessary

Features of the top-level control

• Fast: Less than a second per problem

• General: Independent of specific modules

• Allows participation of a human expert: Initial knowledge and interactive advice

Features of the automatic control

• Several learning mechanisms

• Symbolic and statistical techniques

• Combining exploitation and exploration

Automatic control

Solve the problem or learn additional knowledge?

Automatic control

Which learner to apply?Which past results to use?

Solve the problem or learn additional knowledge?

Automatic control

Which learner to apply?Which past results to use?

Invoke the selectedlearning module

Solve the problem or learn additional knowledge?

Automatic control

Which learner to apply?Which past results to use?

Solve or skip the problem?With which search module?Which learned data to use?

Invoke the selectedlearning module

Solve the problem or learn additional knowledge?

Automatic control

Which learner to apply?Which past results to use?

Solve or skip the problem?With which search module?Which learned data to use?

Invoke the selectedlearning module

Invoke the selectedsearch module

Wait for the next problem

Solve the problem or learn additional knowledge?

Automatic control

Which learner to apply?Which past results to use?

Solve or skip the problem?With which search module?Which learned data to use?

Invoke the selectedlearning module

Invoke the selectedsearch module

Wait for the next problem

Solve the problem or learn additional knowledge?

Performance exampleSolving a series of 50 problems, and learning which modules are the best.

order of solving problems

Performance exampleSolving a series of 50 problems, and learning which modules are the best.

order of solving problems

Performance exampleSolving a series of 500 problems, and learning which modules are the best.

order of solving problems

Applications

• Past: AI planning (Prodigy)

• Present: Hardware design (power estimates)

• Future: Biomedical data, vision, ...